Nov 01 22:19:44.938853 ip-10-0-141-148 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Nov 01 22:19:44.938866 ip-10-0-141-148 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Nov 01 22:19:44.938875 ip-10-0-141-148 systemd[1]: kubelet.service: Failed with result 'resources'.
Nov 01 22:19:44.939379 ip-10-0-141-148 systemd[1]: Failed to start Kubernetes Kubelet.
Nov 01 22:19:54.755384 ip-10-0-141-148 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Nov 01 22:19:54.755400 ip-10-0-141-148 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c6fb49febb804333986507fa678b8f26 --
Nov 01 22:21:40.133783 ip-10-0-141-148 systemd[1]: Starting Kubernetes Kubelet...
Nov 01 22:21:40.594051 ip-10-0-141-148 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 22:21:40.594051 ip-10-0-141-148 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 01 22:21:40.594051 ip-10-0-141-148 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 22:21:40.594051 ip-10-0-141-148 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 22:21:40.597755 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597674    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.597755 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597755    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.597755 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597759    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597763    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597766    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597769    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597772    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597774    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597777    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597781    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597783    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597786    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597789    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597792    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597795    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597797    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597800    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597802    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597805    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597808    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597810    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597813    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.597852 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597816    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597818    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597821    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597824    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597827    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597830    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597841    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597844    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597847    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597850    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597852    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597855    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597859    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597864    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597868    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597872    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597875    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597878    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597881    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.598316 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597884    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597887    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597889    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597892    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597894    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597905    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597909    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597912    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597915    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597917    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597920    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597923    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597926    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597929    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597931    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597934    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597937    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597940    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597943    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597946    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.598773 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597949    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597952    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597955    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597958    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597960    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597963    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597967    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597970    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597974    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597978    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597981    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597984    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597987    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597989    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597992    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597995    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.597998    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598000    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598005    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.599234 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598009    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598011    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598014    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598018    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598021    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598024    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598027    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598030    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598033    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598036    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598039    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598042    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598044    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598047    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598050    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598053    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598056    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598058    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598061    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.599690 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598063    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598066    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598070    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598072    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598075    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598077    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598080    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598906    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598913    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598916    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598919    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598922    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598925    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598928    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598930    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598934    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598937    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598940    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598943    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.600217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598945    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598948    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598951    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598953    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598956    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598959    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598961    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598964    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598967    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598970    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598972    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598975    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598978    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598981    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598983    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598986    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598988    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598992    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.598996    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.601051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599000    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599002    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599005    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599008    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599011    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599014    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599016    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599019    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599021    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599024    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599027    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599030    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599032    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599035    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599038    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599041    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599043    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599046    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599048    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599051    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.601782 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599053    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599056    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599059    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599062    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599065    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599067    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599070    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599072    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599075    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599078    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599080    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599083    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599085    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599088    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599090    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599093    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599096    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599099    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599102    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.602287 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599105    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599107    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599110    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599113    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599115    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599119    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599122    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599125    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599127    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599130    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599134    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599138    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599141    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599143    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599146    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599149    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599151    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599154    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599157    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.602772 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599160    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599162    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599165    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599167    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599170    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599172    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599175    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599177    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599180    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599183    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599186    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599189    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599191    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599194    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599197    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599199    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599203    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599704    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599710    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599713    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.603217 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599716    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599719    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599722    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599725    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599728    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599731    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599733    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599736    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599741    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599744    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599747    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599750    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599754    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599757    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599760    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599764    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599767    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599770    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599773    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599775    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.603703 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599778    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599781    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599783    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599786    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599788    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599791    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599794    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599796    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599799    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599803    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599807    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599811    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599813    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599816    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599818    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599821    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599824    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599826    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599829    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.604184 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599831    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599835    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599838    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599840    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599843    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599847    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599850    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599853    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599855    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599858    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599860    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599863    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599865    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599868    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599871    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599873    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599883    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599886    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599889    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.604655 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599891    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599894    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599896    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599899    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599902    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599906    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599908    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599911    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599914    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599916    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599919    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599921    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599924    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599926    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599931    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599933    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599936    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599938    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599943    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599946    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.605094 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599948    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599951    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599953    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599956    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599959    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599961    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599964    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599966    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599969    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599972    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599974    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599977    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599979    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599982    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599985    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599987    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599990    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599992    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599995    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.599998    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.605625 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.600001    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.600003    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.600006    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.600009    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.600011    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600094    2584 flags.go:64] FLAG: --address="0.0.0.0"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600103    2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600109    2584 flags.go:64] FLAG: --anonymous-auth="true"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600114    2584 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600118    2584 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600125    2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600132    2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600138    2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600143    2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600148    2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600154    2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600157    2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600161    2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600164    2584 flags.go:64] FLAG: --cgroup-root=""
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600167    2584 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600170    2584 flags.go:64] FLAG: --client-ca-file=""
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600173    2584 flags.go:64] FLAG: --cloud-provider="external"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600176    2584 flags.go:64] FLAG: --cluster-dns="[]"
Nov 01 22:21:40.606109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600181    2584 flags.go:64] FLAG: --cluster-domain=""
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600184    2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600187    2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600190    2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600194    2584 flags.go:64] FLAG: --container-log-max-files="5"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600198    2584 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600201    2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600204    2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600208    2584 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600211    2584 flags.go:64] FLAG: --contention-profiling="false"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600214    2584 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600217    2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600220    2584 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600224    2584 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600228    2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600232    2584 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600235    2584 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600238    2584 flags.go:64] FLAG: --enable-load-reader="false"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600241    2584 flags.go:64] FLAG: --enable-server="true"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600246    2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600249    2584 flags.go:64] FLAG: --event-burst="100"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600253    2584 flags.go:64] FLAG: --event-qps="50"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600256    2584 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600260    2584 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 01 22:21:40.606661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600263    2584 flags.go:64] FLAG: --eviction-hard=""
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600267    2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600271    2584 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600274    2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600277    2584 flags.go:64] FLAG: --eviction-soft=""
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600280    2584 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600283    2584 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600286    2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600289    2584 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600292    2584 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600295    2584 flags.go:64] FLAG: --fail-swap-on="true"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600298    2584 flags.go:64] FLAG: --feature-gates=""
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600302    2584 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600305    2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600308    2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600311    2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600314    2584 flags.go:64] FLAG: --healthz-port="10248"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600318    2584 flags.go:64] FLAG: --help="false"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600321    2584 flags.go:64] FLAG: --hostname-override="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600324    2584 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600327    2584 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600330    2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600333    2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 01 22:21:40.607247 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600337    2584 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600341    2584 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600344    2584 flags.go:64] FLAG: --image-service-endpoint=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600347    2584 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600351    2584 flags.go:64] FLAG: --kube-api-burst="100"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600354    2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600357    2584 flags.go:64] FLAG: --kube-api-qps="50"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600403    2584 flags.go:64] FLAG: --kube-reserved=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600909    2584 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600919    2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600925    2584 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600931    2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600937    2584 flags.go:64] FLAG: --lock-file=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600942    2584 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600948    2584 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600954    2584 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600963    2584 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600975    2584 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600980    2584 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600985    2584 flags.go:64] FLAG: --logging-format="text"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600990    2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.600996    2584 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601000    2584 flags.go:64] FLAG: --manifest-url=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601005    2584 flags.go:64] FLAG: --manifest-url-header=""
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601013    2584 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 01 22:21:40.607800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601022    2584 flags.go:64] FLAG: --max-open-files="1000000"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601029    2584 flags.go:64] FLAG: --max-pods="110"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601035    2584 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601040    2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601045    2584 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601049    2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601054    2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601060    2584 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601069    2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601082    2584 flags.go:64] FLAG: --node-status-max-images="50"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601087    2584 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601093    2584 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601098    2584 flags.go:64] FLAG: --pod-cidr=""
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601103    2584 flags.go:64] FLAG: --pod-manifest-path=""
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601108    2584 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601125    2584 flags.go:64] FLAG: --pods-per-core="0"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601134    2584 flags.go:64] FLAG: --port="10250"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601139    2584 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601144    2584 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0fbe7c4503bda3241"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601149    2584 flags.go:64] FLAG: --qos-reserved=""
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601154    2584 flags.go:64] FLAG: --read-only-port="10255"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601159    2584 flags.go:64] FLAG: --register-node="true"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601164    2584 flags.go:64] FLAG: --register-with-taints=""
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601170    2584 flags.go:64] FLAG: --registry-burst="10"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601179    2584 flags.go:64] FLAG: --registry-qps="5"
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601184    2584 flags.go:64] FLAG: --reserved-cpus=""
Nov 01 22:21:40.608570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601189    2584 flags.go:64] FLAG: --reserved-memory=""
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601195    2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601200    2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601205    2584 flags.go:64] FLAG: --rotate-certificates="false"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601210    2584 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601215    2584 flags.go:64] FLAG: --runonce="false"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601224    2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601230    2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601235    2584 flags.go:64] FLAG: --seccomp-default="false"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601240    2584 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601244    2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601250    2584 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601255    2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601260    2584 flags.go:64] FLAG: --storage-driver-password="root"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601265    2584 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601274    2584 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601279    2584 flags.go:64] FLAG: --storage-driver-user="root"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601283    2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601288    2584 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601293    2584 flags.go:64] FLAG: --system-cgroups=""
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601298    2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601309    2584 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601320    2584 flags.go:64] FLAG: --tls-cert-file=""
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601326    2584 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601335    2584 flags.go:64] FLAG: --tls-min-version=""
Nov 01 22:21:40.609722 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601339    2584 flags.go:64] FLAG: --tls-private-key-file=""
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601344    2584 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601348    2584 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601353    2584 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601358    2584 flags.go:64] FLAG: --v="2"
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601371    2584 flags.go:64] FLAG: --version="false"
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601378    2584 flags.go:64] FLAG: --vmodule=""
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601383    2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.601388    2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602079    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602092    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602098    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602101    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602104    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602107    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602110    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602113    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602116    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602120    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602125    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602128    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.610769 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602131    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602134    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602137    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602140    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602143    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602145    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602148    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602151    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602153    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602156    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602159    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602162    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602165    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602167    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602170    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602173    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602176    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602178    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602181    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602184    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.611289 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602190    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602195    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602197    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602200    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602203    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602206    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602208    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602211    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602213    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602216    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602219    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602221    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602224    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602227    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602230    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602232    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602235    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602238    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602241    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.612073 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602244    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602247    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602249    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602252    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602255    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602258    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602260    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602263    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602266    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602268    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602271    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602273    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602276    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602280    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602284    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602286    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602289    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602292    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602295    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602298    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.612915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602301    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602304    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602307    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602310    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602312    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602315    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602318    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602320    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602323    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602326    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602328    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602331    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602334    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602336    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602339    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602342    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602344    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602347    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602350    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.613798 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602352    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602355    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602359    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602363    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602366    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602369    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602374    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602378    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602381    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602384    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602386    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602389    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602392    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602394    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602397    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.614321 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602400    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.602431    2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602476    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602482    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602485    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602488    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602506    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602509    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602512    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602514    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602517    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602520    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602522    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602525    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602528    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602530    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.614945 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602533    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602536    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602539    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602542    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602544    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602547    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602550    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602553    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602559    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602562    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602564    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602567    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602570    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602572    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602575    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602578    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602581    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602583    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602586    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602589    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.615654 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602591    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602594    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602597    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602599    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602602    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602605    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602608    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602610    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602613    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602615    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602618    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602621    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602623    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602626    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602628    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602631    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602633    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602636    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602639    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602641    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.616213 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602646    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602649    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602651    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602654    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602657    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602659    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602662    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602665    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602667    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602670    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602673    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602676    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602678    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602680    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602685    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602688    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602691    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602694    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602697    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.616841 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602699    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602701    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602704    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602707    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602709    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602712    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602714    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602717    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602719    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602722    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602724    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602727    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602730    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602735    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602737    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602740    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602742    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602745    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602749    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.617352 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602753    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602756    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602758    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602761    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602764    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602767    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602769    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602772    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602774    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602777    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602780    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602783    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602785    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.602788    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.610715    2584 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.610734    2584 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.611506    2584 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.611519    2584 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611609    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611618    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.617839 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611624    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611629    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611634    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611638    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611642    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611647    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611652    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611656    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611661    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611665    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611669    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611673    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611677    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611682    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611686    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611691    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611697    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611701    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611705    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.618356 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611710    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611714    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611718    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611722    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611726    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611730    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611735    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611740    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611747    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611753    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611758    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611763    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611767    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611771    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611776    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611780    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611784    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611788    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611793    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.618984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611797    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611801    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611805    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611810    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611814    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611818    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611823    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611827    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611831    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611835    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611840    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611846    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611850    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611854    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611858    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611862    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611867    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611871    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611875    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.619439 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611886    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611890    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611896    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611900    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611905    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611909    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611914    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611918    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611922    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611927    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611931    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611935    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611939    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611943    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611948    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611953    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611957    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611962    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611966    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611970    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.620078 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611974    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611979    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611983    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611987    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.611992    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612000    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612007    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612011    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612015    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612020    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612024    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612029    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612033    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612038    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612043    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612048    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612052    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612057    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612061    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.620891 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612066    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612069    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612074    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612078    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612082    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612085    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612089    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612094    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.612136    2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612362    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612370    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612375    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612380    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612384    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612389    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612393    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.621682 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612397    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612402    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612406    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612411    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612416    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612420    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612424    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612429    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612434    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612438    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612443    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612447    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612452    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612456    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612462    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612469    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612473    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612477    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612482    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.622115 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612487    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612507    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612511    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612515    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612519    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612524    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612528    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612532    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612537    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612541    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612546    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612550    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612555    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612559    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612566    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612572    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612577    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612583    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612587    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612592    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.622623 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612596    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612601    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612605    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612609    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612614    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612618    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612622    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612627    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612631    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612636    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612640    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612644    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612648    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612652    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612656    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612661    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612666    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612670    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612675    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.623106 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612679    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612683    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612687    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612691    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612695    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612699    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612704    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612708    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612712    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612716    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612721    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612726    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612730    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612735    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612739    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612743    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612747    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612751    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612756    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612760    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.623789 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612764    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612768    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612772    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612776    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612780    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612784    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612788    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612792    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612797    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612801    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612806    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612810    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612814    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612818    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612822    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612826    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612831    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612835    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612839    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612844    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.624338 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612848    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.612889    2584 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612940    2584 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612946    2584 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612959    2584 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612965    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612970    2584 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612974    2584 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612979    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612983    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612988    2584 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612991    2584 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.612996    2584 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613000    2584 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613004    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613008    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:21:40.624822 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613013    2584 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613017    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613021    2584 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613025    2584 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613029    2584 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613033    2584 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613038    2584 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613043    2584 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613048    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613052    2584 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613059    2584 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613063    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613068    2584 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613073    2584 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613077    2584 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613081    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613086    2584 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613090    2584 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613094    2584 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:21:40.625214 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613101    2584 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613108    2584 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613113    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613118    2584 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613122    2584 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613126    2584 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613130    2584 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613135    2584 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613139    2584 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613143    2584 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613147    2584 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613152    2584 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613156    2584 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613160    2584 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613165    2584 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613169    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613173    2584 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613177    2584 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613183    2584 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613189    2584 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:21:40.625669 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613195    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613200    2584 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613205    2584 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613212    2584 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613216    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613220    2584 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613225    2584 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613229    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613233    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613239    2584 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613243    2584 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613247    2584 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613251    2584 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613256    2584 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613260    2584 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613265    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613269    2584 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613273    2584 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613277    2584 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613281    2584 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:21:40.626182 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613285    2584 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613290    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613294    2584 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613298    2584 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613302    2584 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613306    2584 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613311    2584 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613315    2584 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613319    2584 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613323    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613327    2584 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613331    2584 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613336    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613340    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613344    2584 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613350    2584 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613354    2584 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613358    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613363    2584 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:21:40.626704 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613367    2584 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613371    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613375    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613380    2584 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613383    2584 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613388    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613392    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613396    2584 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613400    2584 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613405    2584 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613441    2584 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613447    2584 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613453    2584 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:40.613459    2584 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.613553    2584 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.616637    2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.617837    2584 server.go:1014] "Starting client certificate rotation"
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.617945    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 22:21:40.627169 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.617979    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 22:21:40.627730 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.620383    2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 22:21:40.627730 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.623976    2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 22:21:40.638713 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.638689    2584 log.go:25] "Validated CRI v1 runtime API"
Nov 01 22:21:40.642354 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.642338    2584 log.go:25] "Validated CRI v1 image API"
Nov 01 22:21:40.643612 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.643588    2584 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 01 22:21:40.650219 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.650193    2584 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f6eaa64-f5ec-4c28-ab6a-36ee7f609dac:/dev/nvme0n1p4 caa28fe3-5844-416b-951b-3672579012e1:/dev/nvme0n1p3]
Nov 01 22:21:40.650310 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.650219    2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Nov 01 22:21:40.652641 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.652621    2584 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 01 22:21:40.655470 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.655357    2584 manager.go:217] Machine: {Timestamp:2026-11-01 22:21:40.654070101 +0000 UTC m=+0.386899671 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099124 MemoryCapacity:33164328960 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26e013ac02e5adc7f556572b76b41b SystemUUID:ec26e013-ac02-e5ad-c7f5-56572b76b41b BootID:c6fb49fe-bb80-4333-9865-07fa678b8f26 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:d0:d0:3a:d7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:d0:d0:3a:d7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f6:a8:61:a4:b2:25 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164328960 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 01 22:21:40.655470 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.655469    2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 01 22:21:40.655624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.655607    2584 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 01 22:21:40.656819 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.656788    2584 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 01 22:21:40.656983 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.656821    2584 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-148.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 01 22:21:40.657025 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.656985    2584 topology_manager.go:143] "Creating topology manager with none policy"
Nov 01 22:21:40.657025 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.656993    2584 container_manager_linux.go:311] "Creating device plugin manager"
Nov 01 22:21:40.657083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.657037    2584 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 22:21:40.657083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.657055    2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 22:21:40.658298 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.658289    2584 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 01 22:21:40.662563 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662551    2584 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 01 22:21:40.662674 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662665    2584 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 01 22:21:40.662703 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662697    2584 kubelet.go:493] "Attempting to sync node with API server"
Nov 01 22:21:40.662733 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662711    2584 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 01 22:21:40.662733 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662721    2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 01 22:21:40.662733 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662730    2584 kubelet.go:405] "Adding apiserver pod source"
Nov 01 22:21:40.662810 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.662740    2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 01 22:21:40.663075 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.663062    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 22:21:40.665603 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.665586    2584 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 01 22:21:40.671986 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.671973    2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 22:21:40.672911 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.672898    2584 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 01 22:21:40.672954 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.672929    2584 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 01 22:21:40.673081 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673072    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 01 22:21:40.673113 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673087    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 01 22:21:40.673113 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673094    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 01 22:21:40.673113 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673099    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 01 22:21:40.673113 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673105    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 01 22:21:40.673113 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673111    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 01 22:21:40.673826 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673817    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 01 22:21:40.673860 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673828    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 01 22:21:40.673860 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673835    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 01 22:21:40.673860 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673842    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 01 22:21:40.673860 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673850    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 01 22:21:40.673860 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.673859    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 01 22:21:40.674664 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.674654    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 01 22:21:40.674664 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.674664    2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 01 22:21:40.678716 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.678697    2584 server.go:1263] "Started kubelet"
Nov 01 22:21:40.678876 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.678840    2584 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 01 22:21:40.678876 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.678838    2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 01 22:21:40.678979 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.678906    2584 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 01 22:21:40.679514 ip-10-0-141-148 systemd[1]: Started Kubernetes Kubelet.
Nov 01 22:21:40.680202 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.680106    2584 server.go:317] "Adding debug handlers to kubelet server"
Nov 01 22:21:40.680452 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.680431    2584 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 01 22:21:40.685313 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.685287    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-57h7v"
Nov 01 22:21:40.686188 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.686162    2584 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Nov 01 22:21:40.686423 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.686408    2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-148.us-east-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Nov 01 22:21:40.686514 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.686459    2584 reflector.go:204] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-148.us-east-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Node"
Nov 01 22:21:40.686672 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.686643    2584 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service"
Nov 01 22:21:40.689961 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.686310    2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-148.us-east-2.compute.internal.18e40bf58225207f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-148.us-east-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-148.us-east-2.compute.internal,},FirstTimestamp:2026-11-01 22:21:40.678664319 +0000 UTC m=+0.411493890,LastTimestamp:2026-11-01 22:21:40.678664319 +0000 UTC m=+0.411493890,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-148.us-east-2.compute.internal,}"
Nov 01 22:21:40.690823 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.690803    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-57h7v"
Nov 01 22:21:40.695168 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.695149    2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 01 22:21:40.695794 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.695771    2584 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 01 22:21:40.696510 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696478    2584 factory.go:55] Registering systemd factory
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696514    2584 factory.go:223] Registration of the systemd container factory successfully
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696652    2584 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696668    2584 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696742    2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696791    2584 reconstruct.go:97] "Volume reconstruction finished"
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696795    2584 reconciler.go:29] "Reconciler: start to sync state"
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696858    2584 factory.go:153] Registering CRI-O factory
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696867    2584 factory.go:223] Registration of the crio container factory successfully
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.696872    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696905    2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696918    2584 factory.go:103] Registering Raw factory
Nov 01 22:21:40.697480 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.696928    2584 manager.go:1199] Started watching for new ooms in manager
Nov 01 22:21:40.697983 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.697949    2584 manager.go:319] Starting recovery of all containers
Nov 01 22:21:40.705879 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.705847    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 01 22:21:40.706891 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.706875    2584 manager.go:324] Recovery completed
Nov 01 22:21:40.708863 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.708845    2584 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:21:40.711126 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.711113    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.711906 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.711886    2584 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-148.us-east-2.compute.internal\" not found" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.712960 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.712945    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.713048 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.712976    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.713048 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.712990    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.713817 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.713799    2584 cpu_manager.go:226] "Starting" policy="none"
Nov 01 22:21:40.713817 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.713811    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.713954 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.713812    2584 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 01 22:21:40.713954 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.713859    2584 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 01 22:21:40.714525 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.714507    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.714622 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.714540    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.714622 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.714555    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.715887 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.715866    2584 policy_none.go:50] "Start"
Nov 01 22:21:40.715936 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.715889    2584 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 01 22:21:40.715936 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.715900    2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 01 22:21:40.717885 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.717871    2584 policy_none.go:44] "Start"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.770129    2584 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.771855    2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.771883    2584 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.771905    2584 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.771938    2584 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774400    2584 manager.go:341] "Starting Device Plugin manager"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.774427    2584 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774439    2584 server.go:87] "Starting device plugin registration server"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774680    2584 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774691    2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774800    2584 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774874    2584 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.774883    2584 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.775395    2584 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.775454    2584 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:40.778672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.775475    2584 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:21:40.873110 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.872899    2584 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"]
Nov 01 22:21:40.873219 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.873115    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.874168 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.874150    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.874251 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.874184    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.874251 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.874198    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.875232 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.875213    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.875926 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.875909    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.876003 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.875937    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.876003 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.875948    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.876003 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.875970    2584 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.876441 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.876429    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.876621 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.876607    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.876661 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.876642    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.877149 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.877133    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.877206 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.877165    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.877206 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.877176    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.877274 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.877136    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.877274 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.877243    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.877274 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.877256    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.879406 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.879393    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.879462 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.879420    2584 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:21:40.880112 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.880091    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:21:40.880174 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.880116    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:21:40.880174 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.880130    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:21:40.884128 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.884113    2584 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.884203 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.884134    2584 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-148.us-east-2.compute.internal\": node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:40.898373 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.898355    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.898469 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.898379    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68cec1540c43f1cf2055316a4f5734e4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"68cec1540c43f1cf2055316a4f5734e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.898469 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.898396    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.908441 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.908426    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:40.913949 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.913932    2584 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.916683 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:40.916667    2584 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.998672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.998640    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.998672 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.998670    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.998864 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.998694    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68cec1540c43f1cf2055316a4f5734e4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"68cec1540c43f1cf2055316a4f5734e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.998864 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.998747    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68cec1540c43f1cf2055316a4f5734e4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"68cec1540c43f1cf2055316a4f5734e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.998864 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.998751    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:40.998864 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:40.998751    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:41.008776 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.008751    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.109205 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.109154    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.209981 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.209917    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.219095 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.219071    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:41.221250 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.221232    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:41.310169 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.310131    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.410579 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.410537    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.510715 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.510632    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.611242 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.611211    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.617514 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.617483    2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Nov 01 22:21:41.617669 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.617638    2584 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 01 22:21:41.617669 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.617638    2584 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Nov 01 22:21:41.696462 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.696410    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-31 22:16:40 +0000 UTC" deadline="2028-07-03 11:56:52.323601782 +0000 UTC"
Nov 01 22:21:41.696462 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.696456    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14629h35m10.627148658s"
Nov 01 22:21:41.712224 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.712203    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.812407 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.812335    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.913145 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:41.913110    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:41.979220 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:41.979176    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68cec1540c43f1cf2055316a4f5734e4.slice/crio-9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814 WatchSource:0}: Error finding container 9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814: Status 404 returned error can't find the container with id 9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814
Nov 01 22:21:41.984051 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:41.984027    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod564eced2d21b536583a4d40aec7fcb5f.slice/crio-c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032 WatchSource:0}: Error finding container c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032: Status 404 returned error can't find the container with id c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032
Nov 01 22:21:41.986299 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:41.986279    2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 22:21:42.013771 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.013743    2584 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-141-148.us-east-2.compute.internal\" not found"
Nov 01 22:21:42.096791 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.096717    2584 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:21:42.102615 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.102595    2584 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:21:42.198922 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.198886    2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:42.212814 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.212789    2584 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 22:21:42.214026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.214008    2584 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:21:42.223295 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.223272    2584 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 22:21:42.580396 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.580325    2584 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:21:42.664721 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.664427    2584 apiserver.go:52] "Watching apiserver"
Nov 01 22:21:42.672763 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.672729    2584 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 01 22:21:42.673092 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.673070    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-9s2pz","kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd","openshift-cluster-node-tuning-operator/tuned-6zmgx","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal","openshift-multus/multus-wr5zt","openshift-multus/network-metrics-daemon-6gpsh","kube-system/konnectivity-agent-b9n6q","openshift-dns/node-resolver-9xfht","openshift-multus/multus-additional-cni-plugins-zx4lx","openshift-image-registry/node-ca-vjxrr","openshift-network-diagnostics/network-check-target-lg7nw","openshift-network-operator/iptables-alerter-nwqgx","openshift-ovn-kubernetes/ovnkube-node-h5gq2"]
Nov 01 22:21:42.675593 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.675569    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.678612 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.678592    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.679869 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.679846    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.680659    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.680861    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-4hpp8\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.680888    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.681100    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.681708    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.681944    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-79k59\""
Nov 01 22:21:42.682087 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.681999    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.683298 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.683278    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:42.683393 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.683351    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:42.685518 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.685484    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.687813 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.687792    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.691429 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.690098    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.691429 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.690175    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:42.693186 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.692699    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.695710 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.695691    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.696556 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.696542    2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Nov 01 22:21:42.697782 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.697763    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.698048 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.698029    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.698177 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.698164    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.698246 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.698206    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 01 22:21:42.698246 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.698227    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.698330 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.698256    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.700209 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.699980    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:42.700209 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.700040    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:42.704408 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.704168    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.706747 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.706727    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.707927 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.707906    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-host\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.708012 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.707943    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-multus\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708012 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.707969    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:42.708012 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.707993    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c80d6c27-cb56-46c7-bed5-b96e23274d9c-agent-certs\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.708166 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708019    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.708166 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708042    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-k8s-cni-cncf-io\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708166 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708065    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cnibin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708166 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708111    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-hosts-file\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.708166 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708137    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.708166 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708161    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-dbus\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708188    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-socket-dir-parent\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708210    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-bin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708233    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-kubelet\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708254    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-host\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708276    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-socket-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708297    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-netns\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708319    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-daemon-config\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708343    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-tmp-dir\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708366    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-serviceca\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708395    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cni-binary-copy\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708423    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-sys-fs\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708449    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-multus-certs\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.708516 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysconfig\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708816    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-var-lib-kubelet\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708847    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-tmp\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708880    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708902    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708934    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fhbf\" (UniqueName: \"kubernetes.io/projected/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-kube-api-access-6fhbf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708957    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldv8d\" (UniqueName: \"kubernetes.io/projected/b546a0cd-894e-46ab-b097-6684578a546c-kube-api-access-ldv8d\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.708986    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98blt\" (UniqueName: \"kubernetes.io/projected/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-kube-api-access-98blt\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709009    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c80d6c27-cb56-46c7-bed5-b96e23274d9c-konnectivity-ca\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709032    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-system-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709055    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.709107 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709093    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709115    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-tuned\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709160    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-cnibin\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709185    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-etc-kubernetes\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709206    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-kubernetes\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709241    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-conf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-kubelet-config\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709288    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-system-cni-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709309    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-run\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709332    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709364    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwpgl\" (UniqueName: \"kubernetes.io/projected/298f62f9-b298-474c-adec-6b780cca9db8-kube-api-access-nwpgl\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709395    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-hostroot\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709415    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-sys\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709438    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swmmr\" (UniqueName: \"kubernetes.io/projected/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-kube-api-access-swmmr\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709462    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-modprobe-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.709624 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709485    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-lib-modules\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709524    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jstc\" (UniqueName: \"kubernetes.io/projected/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-kube-api-access-2jstc\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709652    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709687    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709724    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-conf-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709749    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-systemd\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-registration-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709792    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-device-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.710243 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.709810    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96spj\" (UniqueName: \"kubernetes.io/projected/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-kube-api-access-96spj\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:42.710627 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.710433    2584 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Nov 01 22:21:42.726236 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.726214    2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r82b8"
Nov 01 22:21:42.733453 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.733432    2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r82b8"
Nov 01 22:21:42.780443 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.780388    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"564eced2d21b536583a4d40aec7fcb5f","Type":"ContainerStarted","Data":"c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032"}
Nov 01 22:21:42.780675 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.780427    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zwl5x\""
Nov 01 22:21:42.780765 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.780742    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vvcmq\""
Nov 01 22:21:42.780820 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.780767    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 01 22:21:42.780820 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.780798    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 01 22:21:42.780916 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.780845    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vv5lb\""
Nov 01 22:21:42.781743 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.781722    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-p26bd\""
Nov 01 22:21:42.781908 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.781808    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 01 22:21:42.781908 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.781853    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j5zd8\""
Nov 01 22:21:42.782027 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.781917    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dclh4\""
Nov 01 22:21:42.782027 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.781962    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qjk9v\""
Nov 01 22:21:42.782876 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.782852    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"68cec1540c43f1cf2055316a4f5734e4","Type":"ContainerStarted","Data":"9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814"}
Nov 01 22:21:42.797393 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.797370    2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 01 22:21:42.798588 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798566    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 01 22:21:42.798588 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798570    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 01 22:21:42.798700 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798595    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.798700 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798623    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 01 22:21:42.798700 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798685    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.798700 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798690    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 01 22:21:42.798700 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798696    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.798836 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798704    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.798836 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798727    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 01 22:21:42.798836 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798750    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 01 22:21:42.798836 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.798789    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 01 22:21:42.800053 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.800033    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 01 22:21:42.800150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.800136    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 01 22:21:42.810383 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810361    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovn-node-metrics-cert\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.810484 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-system-cni-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.810484 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810436    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-run\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.810484 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810462    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.810648 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810506    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.810648 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810536    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-env-overrides\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.810648 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810582    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-system-cni-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.810648 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810595    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwpgl\" (UniqueName: \"kubernetes.io/projected/298f62f9-b298-474c-adec-6b780cca9db8-kube-api-access-nwpgl\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810685    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810704    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-hostroot\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810708    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-run\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810737    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-sys\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810741    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-hostroot\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810768    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4njbt\" (UniqueName: \"kubernetes.io/projected/cc7341bb-795d-430a-a015-36004ec6723a-kube-api-access-4njbt\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810791    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-netns\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810799    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-sys\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.810818 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810808    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-var-lib-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810835    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ms7s9\" (UniqueName: \"kubernetes.io/projected/cf01e723-91f6-4c22-8c8a-78b1cdd730af-kube-api-access-ms7s9\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810872    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swmmr\" (UniqueName: \"kubernetes.io/projected/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-kube-api-access-swmmr\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810905    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-modprobe-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.810971    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-lib-modules\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811023    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jstc\" (UniqueName: \"kubernetes.io/projected/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-kube-api-access-2jstc\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811051    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811065    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-modprobe-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811133    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-conf-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811147    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-lib-modules\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811163    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-systemd\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811278    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-conf-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811293    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-registration-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811346    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811342    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-systemd\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811407    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-device-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811435    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-registration-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.811559 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811513    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96spj\" (UniqueName: \"kubernetes.io/projected/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-kube-api-access-96spj\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811566    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-host\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811528    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-device-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811656    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-systemd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811673    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-host\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811705    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811710    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-etc-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811727    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-multus\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811774    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-multus\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.811792 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811776    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811802    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c80d6c27-cb56-46c7-bed5-b96e23274d9c-agent-certs\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811843    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811868    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-k8s-cni-cncf-io\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.811883    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811892    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-kubelet\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811970    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-k8s-cni-cncf-io\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.811983    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-slash\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.812013    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:21:43.311994918 +0000 UTC m=+3.044824498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812041    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-ovn\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.812129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812117    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cnibin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812142    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-hosts-file\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812166    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812182    2584 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812192    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-dbus\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812209    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-hosts-file\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812224    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812269    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-bin\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812194    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cnibin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812279    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812282    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-dbus\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812307    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-socket-dir-parent\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812309    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812332    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-bin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812355    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-kubelet\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812361    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-socket-dir-parent\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812386    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-kubelet\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812391    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-host\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.812576 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812413    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-bin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812454    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-host\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-socket-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812508    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-netns\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812524    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-daemon-config\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812559    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-netns\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812562    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-tmp-dir\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812601    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-serviceca\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812623    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-systemd-units\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812624    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-socket-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812646    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-netd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cni-binary-copy\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812774    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-sys-fs\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812808    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7341bb-795d-430a-a015-36004ec6723a-iptables-alerter-script\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812845    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812873    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-multus-certs\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812896    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysconfig\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.813442 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812919    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-var-lib-kubelet\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.812954    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-tmp\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813060    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-node-log\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813088    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813098    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-serviceca\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813100    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-tmp-dir\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813112    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813161    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fhbf\" (UniqueName: \"kubernetes.io/projected/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-kube-api-access-6fhbf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813186    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-daemon-config\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813237    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813292    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-multus-certs\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813292    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldv8d\" (UniqueName: \"kubernetes.io/projected/b546a0cd-894e-46ab-b097-6684578a546c-kube-api-access-ldv8d\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813378    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98blt\" (UniqueName: \"kubernetes.io/projected/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-kube-api-access-98blt\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813406    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7341bb-795d-430a-a015-36004ec6723a-host-slash\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813431    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-config\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813487    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c80d6c27-cb56-46c7-bed5-b96e23274d9c-konnectivity-ca\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813561    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-sys-fs\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.814239 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813557    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cni-binary-copy\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813564    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-system-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813617    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813663    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-system-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813673    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-log-socket\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813702    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-script-lib\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813709    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysconfig\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813729    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-tuned\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.813786    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813792    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.813832    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret podName:304f876e-02df-45da-af55-62cae8468a50 nodeName:}" failed. No retries permitted until 2026-11-01 22:21:43.313816095 +0000 UTC m=+3.046645652 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret") pod "global-pull-secret-syncer-9s2pz" (UID: "304f876e-02df-45da-af55-62cae8468a50") : object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813884    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-cnibin\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813914    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-etc-kubernetes\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813937    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-kubernetes\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.813961    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-conf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814019    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-kubelet-config\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.814840 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814068    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c80d6c27-cb56-46c7-bed5-b96e23274d9c-konnectivity-ca\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814084    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-kubelet-config\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814107    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814118    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-cnibin\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814136    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-conf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814153    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-kubernetes\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814143    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-var-lib-kubelet\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814165    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-etc-kubernetes\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.815460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.814238    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.817307 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.817286    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-tmp\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.817391 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.817313    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c80d6c27-cb56-46c7-bed5-b96e23274d9c-agent-certs\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:42.819111 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.819088    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-tuned\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.819850 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.819831    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 22:21:42.819928 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.819857    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 22:21:42.819928 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.819869    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqjkn for pod openshift-network-diagnostics/network-check-target-lg7nw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:42.820044 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:42.819929    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn podName:2bfbadae-7483-48c4-9fe4-a178c2e3c15b nodeName:}" failed. No retries permitted until 2026-11-01 22:21:43.319912745 +0000 UTC m=+3.052742322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lqjkn" (UniqueName: "kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn") pod "network-check-target-lg7nw" (UID: "2bfbadae-7483-48c4-9fe4-a178c2e3c15b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:42.822187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.822159    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98blt\" (UniqueName: \"kubernetes.io/projected/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-kube-api-access-98blt\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:42.823358 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.823335    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldv8d\" (UniqueName: \"kubernetes.io/projected/b546a0cd-894e-46ab-b097-6684578a546c-kube-api-access-ldv8d\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:42.823814 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.823792    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jstc\" (UniqueName: \"kubernetes.io/projected/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-kube-api-access-2jstc\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:42.824028 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.824000    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96spj\" (UniqueName: \"kubernetes.io/projected/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-kube-api-access-96spj\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:42.825260 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.825238    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swmmr\" (UniqueName: \"kubernetes.io/projected/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-kube-api-access-swmmr\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:42.825925 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.825778    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwpgl\" (UniqueName: \"kubernetes.io/projected/298f62f9-b298-474c-adec-6b780cca9db8-kube-api-access-nwpgl\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:42.828017 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.827985    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fhbf\" (UniqueName: \"kubernetes.io/projected/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-kube-api-access-6fhbf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:42.846281 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.846194    2584 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:21:42.914952 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.914918    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-kubelet\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.914952 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.914952    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-slash\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.914973    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-ovn\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.914996    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915022    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-bin\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915029    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-kubelet\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915084    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-bin\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915101    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-slash\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915189 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915162    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-ovn\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915203    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915240    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-systemd-units\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915266    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-netd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915288    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-systemd-units\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915297    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7341bb-795d-430a-a015-36004ec6723a-iptables-alerter-script\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915323    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-netd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915323    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915371    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915390    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-node-log\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915430    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7341bb-795d-430a-a015-36004ec6723a-host-slash\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915446    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-node-log\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915456    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-config\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915480    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7341bb-795d-430a-a015-36004ec6723a-host-slash\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915512    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-log-socket\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.915545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915536    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-script-lib\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915578    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovn-node-metrics-cert\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915607    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915633    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-env-overrides\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njbt\" (UniqueName: \"kubernetes.io/projected/cc7341bb-795d-430a-a015-36004ec6723a-kube-api-access-4njbt\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915684    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-netns\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915710    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-var-lib-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915733    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms7s9\" (UniqueName: \"kubernetes.io/projected/cf01e723-91f6-4c22-8c8a-78b1cdd730af-kube-api-access-ms7s9\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915766    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-systemd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915789    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-etc-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915856    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-etc-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915903    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.915610    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-log-socket\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916008    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7341bb-795d-430a-a015-36004ec6723a-iptables-alerter-script\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916116    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-config\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916187 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916184    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-systemd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916719 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916215    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-netns\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916719 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916236    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-var-lib-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916719 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916269    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-script-lib\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.916719 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.916277    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-env-overrides\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.918158 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.918133    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovn-node-metrics-cert\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.923169 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.923142    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njbt\" (UniqueName: \"kubernetes.io/projected/cc7341bb-795d-430a-a015-36004ec6723a-kube-api-access-4njbt\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:42.923649 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.923630    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms7s9\" (UniqueName: \"kubernetes.io/projected/cf01e723-91f6-4c22-8c8a-78b1cdd730af-kube-api-access-ms7s9\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:42.993547 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:42.993511    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wr5zt"
Nov 01 22:21:43.001295 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.001268    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:21:43.012175 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.012142    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295d3d68_c2d1_497f_9a6b_7bd5017b2a41.slice/crio-c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb WatchSource:0}: Error finding container c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb: Status 404 returned error can't find the container with id c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb
Nov 01 22:21:43.012948 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.012923    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:21:43.021908 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.021886    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:21:43.022531 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.022510    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf06a906_365c_44b3_b3c2_75a74fc8fdf0.slice/crio-f0ed8e99e087aa3013f90b8181de332546fdc53c43e8e3bfcb9523f8e163b83d WatchSource:0}: Error finding container f0ed8e99e087aa3013f90b8181de332546fdc53c43e8e3bfcb9523f8e163b83d: Status 404 returned error can't find the container with id f0ed8e99e087aa3013f90b8181de332546fdc53c43e8e3bfcb9523f8e163b83d
Nov 01 22:21:43.030345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.030323    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:43.035516 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.031996    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb546a0cd_894e_46ab_b097_6684578a546c.slice/crio-6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea WatchSource:0}: Error finding container 6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea: Status 404 returned error can't find the container with id 6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea
Nov 01 22:21:43.043850 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.043813    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80d6c27_cb56_46c7_bed5_b96e23274d9c.slice/crio-098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9 WatchSource:0}: Error finding container 098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9: Status 404 returned error can't find the container with id 098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9
Nov 01 22:21:43.053507 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.053471    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:21:43.055682 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.055660    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:21:43.058421 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.058398    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:21:43.072195 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.072164    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7fb00c_4f60_4b4c_9402_65cc6ec7e887.slice/crio-c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3 WatchSource:0}: Error finding container c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3: Status 404 returned error can't find the container with id c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3
Nov 01 22:21:43.073387 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.073355    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc7341bb_795d_430a_a015_36004ec6723a.slice/crio-a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf WatchSource:0}: Error finding container a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf: Status 404 returned error can't find the container with id a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf
Nov 01 22:21:43.083606 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.083585    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:21:43.093677 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:21:43.093647    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf01e723_91f6_4c22_8c8a_78b1cdd730af.slice/crio-6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463 WatchSource:0}: Error finding container 6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463: Status 404 returned error can't find the container with id 6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463
Nov 01 22:21:43.319157 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.319122    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:43.319327 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.319203    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:43.319327 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.319295    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:43.319327 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.319311    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:43.319439 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.319364    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret podName:304f876e-02df-45da-af55-62cae8468a50 nodeName:}" failed. No retries permitted until 2026-11-01 22:21:44.319349725 +0000 UTC m=+4.052179286 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret") pod "global-pull-secret-syncer-9s2pz" (UID: "304f876e-02df-45da-af55-62cae8468a50") : object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:43.319439 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.319385    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:21:44.319375285 +0000 UTC m=+4.052204857 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:43.420350 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.420271    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:43.420539 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.420412    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 22:21:43.420539 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.420433    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 22:21:43.420539 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.420445    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqjkn for pod openshift-network-diagnostics/network-check-target-lg7nw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:43.420539 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.420522    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn podName:2bfbadae-7483-48c4-9fe4-a178c2e3c15b nodeName:}" failed. No retries permitted until 2026-11-01 22:21:44.420503745 +0000 UTC m=+4.153333319 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqjkn" (UniqueName: "kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn") pod "network-check-target-lg7nw" (UID: "2bfbadae-7483-48c4-9fe4-a178c2e3c15b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:43.736770 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.736656    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 22:16:42 +0000 UTC" deadline="2028-07-22 19:43:48.153512288 +0000 UTC"
Nov 01 22:21:43.736770 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.736720    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15093h22m4.416796638s"
Nov 01 22:21:43.772452 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.772418    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:43.772674 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:43.772648    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:43.792208 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.792146    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nwqgx" event={"ID":"cc7341bb-795d-430a-a015-36004ec6723a","Type":"ContainerStarted","Data":"a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf"}
Nov 01 22:21:43.810273 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.810049    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b9n6q" event={"ID":"c80d6c27-cb56-46c7-bed5-b96e23274d9c","Type":"ContainerStarted","Data":"098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9"}
Nov 01 22:21:43.818309 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.818220    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd" event={"ID":"b546a0cd-894e-46ab-b097-6684578a546c","Type":"ContainerStarted","Data":"6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea"}
Nov 01 22:21:43.827690 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.827659    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6zmgx" event={"ID":"bf06a906-365c-44b3-b3c2-75a74fc8fdf0","Type":"ContainerStarted","Data":"f0ed8e99e087aa3013f90b8181de332546fdc53c43e8e3bfcb9523f8e163b83d"}
Nov 01 22:21:43.829057 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.828990    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9xfht" event={"ID":"295d3d68-c2d1-497f-9a6b-7bd5017b2a41","Type":"ContainerStarted","Data":"c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb"}
Nov 01 22:21:43.839618 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.839484    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463"}
Nov 01 22:21:43.843153 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.843084    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vjxrr" event={"ID":"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887","Type":"ContainerStarted","Data":"c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3"}
Nov 01 22:21:43.855335 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.855305    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerStarted","Data":"c242fd290495c5f75e5ee2419d953b8fe99310a4f3afa85260c023a03320050c"}
Nov 01 22:21:43.875842 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:43.875808    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wr5zt" event={"ID":"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2","Type":"ContainerStarted","Data":"729125a8c1f724ad90522448c0cedd08e7c96f8d3480af183aff99252aee638b"}
Nov 01 22:21:44.325692 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.325602    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:44.325692 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.325689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:44.325899 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.325829    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:44.325899 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.325890    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:21:46.325873405 +0000 UTC m=+6.058702967 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:44.326293 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.326272    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:44.326388 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.326328    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret podName:304f876e-02df-45da-af55-62cae8468a50 nodeName:}" failed. No retries permitted until 2026-11-01 22:21:46.326312069 +0000 UTC m=+6.059141628 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret") pod "global-pull-secret-syncer-9s2pz" (UID: "304f876e-02df-45da-af55-62cae8468a50") : object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:44.427094 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.426353    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:44.427094 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.426573    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 22:21:44.427094 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.426593    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 22:21:44.427094 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.426606    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqjkn for pod openshift-network-diagnostics/network-check-target-lg7nw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:44.427094 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.426676    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn podName:2bfbadae-7483-48c4-9fe4-a178c2e3c15b nodeName:}" failed. No retries permitted until 2026-11-01 22:21:46.426645716 +0000 UTC m=+6.159475297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqjkn" (UniqueName: "kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn") pod "network-check-target-lg7nw" (UID: "2bfbadae-7483-48c4-9fe4-a178c2e3c15b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:44.737245 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.737147    2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 22:16:42 +0000 UTC" deadline="2028-05-16 12:21:18.612088707 +0000 UTC"
Nov 01 22:21:44.737245 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.737193    2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13477h59m33.874901992s"
Nov 01 22:21:44.783537 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.782668    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:44.783537 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.782909    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:44.783537 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.782959    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:44.783537 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:44.783079    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:44.888714 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:44.888635    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"68cec1540c43f1cf2055316a4f5734e4","Type":"ContainerStarted","Data":"34fd0b2425e3d40fed86bb57f044ae985cef36c2117e575cdf693ff8120c67cc"}
Nov 01 22:21:45.773186 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:45.772895    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:45.773643 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:45.773257    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:46.344475 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.344392    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:46.344654 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.344476    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:46.344654 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.344533    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:46.344654 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.344602    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret podName:304f876e-02df-45da-af55-62cae8468a50 nodeName:}" failed. No retries permitted until 2026-11-01 22:21:50.344583922 +0000 UTC m=+10.077413492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret") pod "global-pull-secret-syncer-9s2pz" (UID: "304f876e-02df-45da-af55-62cae8468a50") : object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:46.344654 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.344618    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:46.344886 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.344682    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:21:50.344664672 +0000 UTC m=+10.077494243 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:46.445781 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.445742    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:46.446073 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.445949    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 22:21:46.446073 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.445966    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 22:21:46.446073 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.445979    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqjkn for pod openshift-network-diagnostics/network-check-target-lg7nw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:46.446073 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.446036    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn podName:2bfbadae-7483-48c4-9fe4-a178c2e3c15b nodeName:}" failed. No retries permitted until 2026-11-01 22:21:50.446018712 +0000 UTC m=+10.178848272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqjkn" (UniqueName: "kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn") pod "network-check-target-lg7nw" (UID: "2bfbadae-7483-48c4-9fe4-a178c2e3c15b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:46.774632 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.774593    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:46.775093 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.774722    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:46.775093 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.774593    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:46.775366 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:46.775214    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:46.905736 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.904811    2584 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="361d92bcb85119ed29fda17dfde4c905f652e3cd20b6c6ba75504c2d4f7f8923" exitCode=0
Nov 01 22:21:46.905736 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.904899    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerDied","Data":"361d92bcb85119ed29fda17dfde4c905f652e3cd20b6c6ba75504c2d4f7f8923"}
Nov 01 22:21:46.913467 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.912840    2584 generic.go:356] "Generic (PLEG): container finished" podID="564eced2d21b536583a4d40aec7fcb5f" containerID="8a51ad3a7cbcc0b3eb39f20c6cb3f57f36ebb1b01dd555c568b26fd950780fac" exitCode=0
Nov 01 22:21:46.913467 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.912888    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"564eced2d21b536583a4d40aec7fcb5f","Type":"ContainerDied","Data":"8a51ad3a7cbcc0b3eb39f20c6cb3f57f36ebb1b01dd555c568b26fd950780fac"}
Nov 01 22:21:46.929525 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:46.928643    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal" podStartSLOduration=4.92862434 podStartE2EDuration="4.92862434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:44.904584191 +0000 UTC m=+4.637413773" watchObservedRunningTime="2026-11-01 22:21:46.92862434 +0000 UTC m=+6.661453920"
Nov 01 22:21:47.772910 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:47.772864    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:47.773099 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:47.773021    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:47.918433 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:47.918369    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"564eced2d21b536583a4d40aec7fcb5f","Type":"ContainerStarted","Data":"1570586369e4050df406c854269e44b0967d1e288b8515873fd65714a507607b"}
Nov 01 22:21:47.921886 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:47.921847    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-b9n6q" event={"ID":"c80d6c27-cb56-46c7-bed5-b96e23274d9c","Type":"ContainerStarted","Data":"6714431828d2c4a4d064fb5cd66ae218252380085f35cdcff806475f443bbd89"}
Nov 01 22:21:47.948511 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:47.948439    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" podStartSLOduration=5.9484188289999995 podStartE2EDuration="5.948418829s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:47.934800965 +0000 UTC m=+7.667630544" watchObservedRunningTime="2026-11-01 22:21:47.948418829 +0000 UTC m=+7.681248410"
Nov 01 22:21:48.775513 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:48.774822    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:48.775513 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:48.774946    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:48.775513 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:48.775296    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:48.775513 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:48.775391    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:48.929542 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:48.929505    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-vjxrr" event={"ID":"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887","Type":"ContainerStarted","Data":"ae31e6f7d59c19ac809f7f1566a4033e40a106ed08fe66fe069df2f5c7199fe1"}
Nov 01 22:21:48.937299 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:48.937269    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd" event={"ID":"b546a0cd-894e-46ab-b097-6684578a546c","Type":"ContainerStarted","Data":"89cc01a64e4043976d1697b2c4ad02729da5418d7d3c0176d7c9e515edee0c12"}
Nov 01 22:21:48.945703 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:48.945261    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-b9n6q" podStartSLOduration=3.922948488 podStartE2EDuration="7.945242016s" totalImagesPullingTime="4.022293528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:47.948932355 +0000 UTC m=+7.681761934" watchObservedRunningTime="2026-11-01 22:21:48.945242016 +0000 UTC m=+8.678071596"
Nov 01 22:21:48.945703 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:48.945525    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-vjxrr" podStartSLOduration=2.44039972 podStartE2EDuration="7.945518587s" totalImagesPullingTime="5.505118867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:48.944667465 +0000 UTC m=+8.677497047" watchObservedRunningTime="2026-11-01 22:21:48.945518587 +0000 UTC m=+8.678348169"
Nov 01 22:21:49.773020 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:49.772479    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:49.773020 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:49.772637    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:50.384602 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.383817    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:50.384602 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.383900    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:50.384602 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.384035    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:50.384602 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.384093    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:21:58.384076533 +0000 UTC m=+18.116906096 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:50.384602 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.384507    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:50.384602 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.384556    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret podName:304f876e-02df-45da-af55-62cae8468a50 nodeName:}" failed. No retries permitted until 2026-11-01 22:21:58.384542011 +0000 UTC m=+18.117371571 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret") pod "global-pull-secret-syncer-9s2pz" (UID: "304f876e-02df-45da-af55-62cae8468a50") : object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:50.485296 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.485219    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:50.486161 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.485666    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 22:21:50.486161 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.485698    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 22:21:50.486161 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.485720    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqjkn for pod openshift-network-diagnostics/network-check-target-lg7nw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:50.486161 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.485777    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn podName:2bfbadae-7483-48c4-9fe4-a178c2e3c15b nodeName:}" failed. No retries permitted until 2026-11-01 22:21:58.485759433 +0000 UTC m=+18.218588996 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqjkn" (UniqueName: "kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn") pod "network-check-target-lg7nw" (UID: "2bfbadae-7483-48c4-9fe4-a178c2e3c15b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:50.486976 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.486888    2584 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 01 22:21:50.768208 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.767389    2584 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:50.768828 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.768598    2584 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:21:50.776265 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.774945    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:50.776265 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.775056    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:50.776265 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.775379    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:50.776265 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:50.775462    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:50.791347 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.790557    2584 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-01T22:21:50.486905874Z","UUID":"3ce22bb3-d2ab-4057-a9c2-c3eb0541d0b0","Handler":null,"Name":"","Endpoint":""}
Nov 01 22:21:50.793100 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.792638    2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 01 22:21:50.793100 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.792667    2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 01 22:21:50.953238 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:50.953164    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd" event={"ID":"b546a0cd-894e-46ab-b097-6684578a546c","Type":"ContainerStarted","Data":"649cb89cf6fe43e663f00fdc60e4b78352a8a4ea34482a1d436d3381ea93af75"}
Nov 01 22:21:51.772514 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:51.772470    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:51.772894 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:51.772614    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:51.957770 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:51.957378    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9xfht" event={"ID":"295d3d68-c2d1-497f-9a6b-7bd5017b2a41","Type":"ContainerStarted","Data":"35525c2ce91dc1d97cb8e3b8e48859a95ec255858b034ebe0a0c7019a7499a12"}
Nov 01 22:21:51.962788 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:51.962737    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerStarted","Data":"fbd8982eb350310d12d2bea5f49a9c50f0546b528fe3bc9fdbfdda08ad86dcb4"}
Nov 01 22:21:51.965202 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:51.965178    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wr5zt" event={"ID":"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2","Type":"ContainerStarted","Data":"170dada9e5ad80b56bdd15d6027f925bc6156d257656cc8bc0d00ca136c5ba07"}
Nov 01 22:21:51.967970 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:51.967948    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd" event={"ID":"b546a0cd-894e-46ab-b097-6684578a546c","Type":"ContainerStarted","Data":"468f76dff8bbe1c200c722138750929ea08afe424ded0da2746f134020267b5d"}
Nov 01 22:21:52.016013 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.015765    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-wr5zt" podStartSLOduration=3.828586666 podStartE2EDuration="12.015748705s" totalImagesPullingTime="8.187162039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:52.015193907 +0000 UTC m=+11.748023488" watchObservedRunningTime="2026-11-01 22:21:52.015748705 +0000 UTC m=+11.748578284"
Nov 01 22:21:52.016230 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.016194    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-9xfht" podStartSLOduration=3.619096904 podStartE2EDuration="12.016187204s" totalImagesPullingTime="8.3970903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:51.992615458 +0000 UTC m=+11.725445037" watchObservedRunningTime="2026-11-01 22:21:52.016187204 +0000 UTC m=+11.749016784"
Nov 01 22:21:52.032287 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.032239    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd" podStartSLOduration=2.6989009189999997 podStartE2EDuration="11.032222116s" totalImagesPullingTime="8.333321197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:52.031663017 +0000 UTC m=+11.764492602" watchObservedRunningTime="2026-11-01 22:21:52.032222116 +0000 UTC m=+11.765051696"
Nov 01 22:21:52.772366 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.772327    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:52.772545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.772378    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:52.772545 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:52.772506    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:52.772982 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:52.772616    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:52.972108 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.972071    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nwqgx" event={"ID":"cc7341bb-795d-430a-a015-36004ec6723a","Type":"ContainerStarted","Data":"8dc7c25b05df9426785fe8a0f9b17e95d42586e780b1a64960b89ba107643254"}
Nov 01 22:21:52.974255 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.974224    2584 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="fbd8982eb350310d12d2bea5f49a9c50f0546b528fe3bc9fdbfdda08ad86dcb4" exitCode=0
Nov 01 22:21:52.975221 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:52.975199    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerDied","Data":"fbd8982eb350310d12d2bea5f49a9c50f0546b528fe3bc9fdbfdda08ad86dcb4"}
Nov 01 22:21:53.772981 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:53.772914    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:53.773581 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:53.773029    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:53.978088 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:53.978058    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-6zmgx" event={"ID":"bf06a906-365c-44b3-b3c2-75a74fc8fdf0","Type":"ContainerStarted","Data":"2925d5de7e12306566a31bdd5ca129d1971eb2705c53160b5f3fcb304f22580f"}
Nov 01 22:21:53.997004 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:53.996959    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-6zmgx" podStartSLOduration=2.5920063669999998 podStartE2EDuration="12.996945704s" totalImagesPullingTime="10.404939337s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:53.996248556 +0000 UTC m=+13.729078137" watchObservedRunningTime="2026-11-01 22:21:53.996945704 +0000 UTC m=+13.729775287"
Nov 01 22:21:54.772985 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:54.772948    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:54.773724 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:54.773085    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:54.773724 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:54.773139    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:54.773724 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:54.773237    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:54.982606 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:54.982571    2584 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="e77fbc384211cb0f09c2517228d61bf075a4ea5c3b073c5a1602bccf809ddd42" exitCode=0
Nov 01 22:21:54.982761 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:54.982652    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerDied","Data":"e77fbc384211cb0f09c2517228d61bf075a4ea5c3b073c5a1602bccf809ddd42"}
Nov 01 22:21:55.003639 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:55.003593    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nwqgx" podStartSLOduration=5.873568864 podStartE2EDuration="14.003579127s" totalImagesPullingTime="8.130010263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:21:54.011159902 +0000 UTC m=+13.743989486" watchObservedRunningTime="2026-11-01 22:21:55.003579127 +0000 UTC m=+14.736408707"
Nov 01 22:21:55.772536 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:55.772488    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:55.772708 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:55.772665    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:56.772337 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:56.772142    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:56.772855 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:56.772151    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:56.772855 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:56.772441    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:56.772855 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:56.772524    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:56.990291 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:56.990258    2584 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="3ff7fc2baf94ee76eebfee53d7124cbe3ad4e7e59b6b99cdfd647062645cc948" exitCode=0
Nov 01 22:21:56.990461 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:56.990303    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerDied","Data":"3ff7fc2baf94ee76eebfee53d7124cbe3ad4e7e59b6b99cdfd647062645cc948"}
Nov 01 22:21:57.772422 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:57.772390    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:57.772908 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:57.772552    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:21:58.440979 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.440780    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:58.441175 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.441014    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:58.441175 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.440934    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:58.441175 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.441109    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.441090188 +0000 UTC m=+34.173919761 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Nov 01 22:21:58.441175 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.441119    2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:58.441175 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.441168    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret podName:304f876e-02df-45da-af55-62cae8468a50 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.441152253 +0000 UTC m=+34.173981820 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret") pod "global-pull-secret-syncer-9s2pz" (UID: "304f876e-02df-45da-af55-62cae8468a50") : object "kube-system"/"original-pull-secret" not registered
Nov 01 22:21:58.541704 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.541669    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:58.541881 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.541814    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Nov 01 22:21:58.541881 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.541835    2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Nov 01 22:21:58.541881 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.541847    2584 projected.go:196] Error preparing data for projected volume kube-api-access-lqjkn for pod openshift-network-diagnostics/network-check-target-lg7nw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:58.541992 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.541907    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn podName:2bfbadae-7483-48c4-9fe4-a178c2e3c15b nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.54189094 +0000 UTC m=+34.274720512 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lqjkn" (UniqueName: "kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn") pod "network-check-target-lg7nw" (UID: "2bfbadae-7483-48c4-9fe4-a178c2e3c15b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Nov 01 22:21:58.773029 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.772954    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:21:58.773430 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.772962    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:21:58.773430 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.773053    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:21:58.773430 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:58.773152    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:21:58.997317 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.997271    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"c64a6b6fff9191c7eae18111e5eb897904fee71edcf3e650efa6f42f637608c4"}
Nov 01 22:21:58.997469 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.997364    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"4a68230d8f495a27c46cf4af9d3a48ab530af4be1eee52d0fe5db82fe483821d"}
Nov 01 22:21:58.997469 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.997415    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"6f7d7819da195125322a8c1b203044843a43b5f287b9e6d3a7116d0b1f0fee0e"}
Nov 01 22:21:58.997469 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.997463    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"10413a80a25b5914ade7c05d32bd15d46574837b4c6ffcf926bb3f1bceb2d5f8"}
Nov 01 22:21:58.997608 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.997525    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"da6bb8bf0bcb0cc31d3ef07bfcaf1f56ac709f000ddcd25ad5c8b7c08b52b5e0"}
Nov 01 22:21:58.997608 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:58.997571    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"bdf4baef874e43da9a4afe349a48e93906c530b8810154d9966a967e04d54804"}
Nov 01 22:21:59.772917 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:21:59.772845    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:21:59.773056 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:21:59.772952    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:00.002348 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:00.002314    2584 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="1a04369ef6e21e4d590fc3e7276b146f86c926d8d18d39477e3f0f1d06135753" exitCode=0
Nov 01 22:22:00.002348 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:00.002353    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerDied","Data":"1a04369ef6e21e4d590fc3e7276b146f86c926d8d18d39477e3f0f1d06135753"}
Nov 01 22:22:00.772901 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:00.772829    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:00.773028 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:00.772917    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:22:00.773028 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:00.773000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:00.773518 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:00.773085    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:22:01.007067 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:01.007032    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"0ef4242f82d29bb2bcc332a3dec391a8d64381732b292b57b1ee4d4fe1f90e63"}
Nov 01 22:22:01.009733 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:01.009706    2584 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="5fb59a8b682125d4f379770c45a36ca1b0323b6979476ff259f709c759ed1a49" exitCode=0
Nov 01 22:22:01.009824 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:01.009754    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerDied","Data":"5fb59a8b682125d4f379770c45a36ca1b0323b6979476ff259f709c759ed1a49"}
Nov 01 22:22:01.772458 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:01.772406    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:01.772666 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:01.772552    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:02.014800 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:02.014766    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" event={"ID":"298f62f9-b298-474c-adec-6b780cca9db8","Type":"ContainerStarted","Data":"6908af1606440615812cc02caba46cbab13c16bd4d56f988eda89a77fd180dda"}
Nov 01 22:22:02.036920 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:02.036843    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zx4lx" podStartSLOduration=9.20286293 podStartE2EDuration="21.036832095s" totalImagesPullingTime="11.833969165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:02.035158528 +0000 UTC m=+21.767988107" watchObservedRunningTime="2026-11-01 22:22:02.036832095 +0000 UTC m=+21.769661675"
Nov 01 22:22:02.772549 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:02.772264    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:02.772686 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:02.772322    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:02.772686 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:02.772650    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:22:02.772775 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:02.772758    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:22:03.020859 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:03.020821    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" event={"ID":"cf01e723-91f6-4c22-8c8a-78b1cdd730af","Type":"ContainerStarted","Data":"b5a7517cb70a0a2b39fcf7facd1d08ad08a38745efe3d704156f267f308b6cbd"}
Nov 01 22:22:03.048090 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:03.048020    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2" podStartSLOduration=7.078973586 podStartE2EDuration="22.048007324s" totalImagesPullingTime="14.969033738s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:03.047426614 +0000 UTC m=+22.780256193" watchObservedRunningTime="2026-11-01 22:22:03.048007324 +0000 UTC m=+22.780836903"
Nov 01 22:22:03.772796 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:03.772767    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:03.772963 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:03.772874    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:04.026950 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.026878    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:04.027485 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.026982    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:04.027485 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.027031    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:04.044162 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.044140    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:04.044287 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.044236    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:04.242859 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.242834    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9s2pz"]
Nov 01 22:22:04.243084 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.242942    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:04.243084 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:04.243027    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:22:04.246785 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.246749    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lg7nw"]
Nov 01 22:22:04.246918 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.246854    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:04.247006 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:04.246976    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:22:04.247546 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.247522    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6gpsh"]
Nov 01 22:22:04.247648 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:04.247634    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:04.247731 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:04.247714    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:05.772288 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:05.772253    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:05.772782 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:05.772357    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:05.772782 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:05.772385    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:05.772782 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:05.772373    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:22:05.772782 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:05.772459    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:22:05.772782 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:05.772532    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:06.974358 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:06.974325    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:06.974754 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:06.974445    2584 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 22:22:06.974877 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:06.974864    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:07.772351 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:07.772321    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:07.772525 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:07.772393    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:07.772525 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:07.772469    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:07.772525 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:07.772477    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:22:07.772655 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:07.772572    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:22:07.772655 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:07.772619    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:09.773108 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:09.773078    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:09.773521 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:09.773177    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:09.773521 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:09.773190    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9s2pz" podUID="304f876e-02df-45da-af55-62cae8468a50"
Nov 01 22:22:09.773521 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:09.773264    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-lg7nw" podUID="2bfbadae-7483-48c4-9fe4-a178c2e3c15b"
Nov 01 22:22:09.773521 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:09.773273    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:09.773521 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:09.773344    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-6gpsh" podUID="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f"
Nov 01 22:22:11.115933 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.115905    2584 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeReady"
Nov 01 22:22:11.116325 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.116024    2584 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Nov 01 22:22:11.147129 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.147100    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rxq5k"]
Nov 01 22:22:11.150214 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.150194    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:11.150968 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.150942    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-r6xqj"]
Nov 01 22:22:11.153979 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.153952    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lkzkr"]
Nov 01 22:22:11.154096 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.154076    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:11.157271 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.157244    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"]
Nov 01 22:22:11.157373 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.157351    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:11.159352 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.159050    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.159352 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.159062    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.161092 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.161073    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"]
Nov 01 22:22:11.161181 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.161112    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.165464 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.165446    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-fvxj8"]
Nov 01 22:22:11.165619 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.165604    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.172010 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.171991    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 01 22:22:11.175614 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.175593    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-8444bb9d4c-m4lpd"]
Nov 01 22:22:11.175712 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.175677    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.187866 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.187839    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"]
Nov 01 22:22:11.187986 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.187967    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.191994 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.191975    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"]
Nov 01 22:22:11.193660 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.192104    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.198440 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.198422    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"]
Nov 01 22:22:11.198584 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.198560    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.203112 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.203096    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:11.203450 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.203425    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"]
Nov 01 22:22:11.209075 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.209055    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-586fc54c5f-9zzlg"]
Nov 01 22:22:11.209185 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.209173    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.211330 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.211311    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-dfcdddc4b-szq25"]
Nov 01 22:22:11.211597 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.211578    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.214880 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.214862    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"]
Nov 01 22:22:11.214969 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.214956    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.218120 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.218103    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"]
Nov 01 22:22:11.218246 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.218231    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.222109 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222092    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rxq5k"]
Nov 01 22:22:11.222196 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222129    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"]
Nov 01 22:22:11.222196 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222153    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lkzkr"]
Nov 01 22:22:11.222196 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222176    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-r6xqj"]
Nov 01 22:22:11.222196 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222197    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"]
Nov 01 22:22:11.222403 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222214    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:11.222529 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222216    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-fvxj8"]
Nov 01 22:22:11.222598 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222559    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8444bb9d4c-m4lpd"]
Nov 01 22:22:11.222598 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222587    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"]
Nov 01 22:22:11.222701 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222608    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"]
Nov 01 22:22:11.222701 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.222637    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-j85jx"]
Nov 01 22:22:11.225774 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.225755    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-fzbm2"]
Nov 01 22:22:11.226049 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.226008    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.229432 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229411    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229438    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229452    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229475    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229513    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dfcdddc4b-szq25"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229536    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-586fc54c5f-9zzlg"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229555    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j85jx"]
Nov 01 22:22:11.229566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229555    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jr28\" (UniqueName: \"kubernetes.io/projected/812cb630-c9f1-4b19-b5e2-b57fda496558-kube-api-access-4jr28\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229576    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fzbm2"]
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229592    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229601    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-config\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229679    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-trusted-ca\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229709    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksq2c\" (UniqueName: \"kubernetes.io/projected/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d-kube-api-access-ksq2c\") pod \"kas-connection-checker-685895c7c9-rxq5k\" (UID: \"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d\") " pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229739    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/812cb630-c9f1-4b19-b5e2-b57fda496558-serving-cert\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229795    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrvmm\" (UniqueName: \"kubernetes.io/projected/42fd8ec3-88cb-4e0b-9d58-1c654ff20582-kube-api-access-vrvmm\") pod \"kas-connection-checker-685895c7c9-lkzkr\" (UID: \"42fd8ec3-88cb-4e0b-9d58-1c654ff20582\") " pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:11.229903 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.229811    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kc4xk\" (UniqueName: \"kubernetes.io/projected/514791bf-52ab-46d1-bec2-44baa1b3f48d-kube-api-access-kc4xk\") pod \"kas-connection-checker-685895c7c9-r6xqj\" (UID: \"514791bf-52ab-46d1-bec2-44baa1b3f48d\") " pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:11.259182 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.259154    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.259693 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.259661    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.259793 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.259697    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 01 22:22:11.260893 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260100    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 01 22:22:11.260893 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260196    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.260893 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260327    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.260893 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260372    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.260893 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260389    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.260893 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260737    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 01 22:22:11.261252 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260954    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.261252 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260971    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.261252 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260105    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.261252 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261198    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 01 22:22:11.261704 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261535    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 01 22:22:11.261704 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261665    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 01 22:22:11.261704 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.260737    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.261865 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261740    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.261963 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261914    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.261963 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261924    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.261963 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.261948    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.262183 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262089    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 01 22:22:11.262183 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262180    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.262268 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262181    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.262268 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262240    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.262362 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262277    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.262362 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262322    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:11.263617 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262702    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 01 22:22:11.263617 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.262750    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 01 22:22:11.270261 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.270226    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278211    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278241    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278241    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278267    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278260    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278285    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278309    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278312    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278286    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5847z\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278280    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278314    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 01 22:22:11.278345 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278340    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-4jwn7\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278325    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278310    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278374    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kglz\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278280    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278360    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278376    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278414    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 01 22:22:11.278768 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.278439    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 01 22:22:11.330261 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330234    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqhph\" (UniqueName: \"kubernetes.io/projected/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-kube-api-access-nqhph\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.330387 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330267    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9918792e-83ee-41de-8794-bf799e14788d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.330387 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330289    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-default-certificate\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.330387 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330343    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jr28\" (UniqueName: \"kubernetes.io/projected/812cb630-c9f1-4b19-b5e2-b57fda496558-kube-api-access-4jr28\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.330570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330387    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81511c2e-f1cb-40ea-8d52-b795feb55690-tmp-dir\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.330570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330511    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.330570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330542    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.330570 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330568    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-trusted-ca\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.330734 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330589    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.330734 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330611    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/71d83910-eb9d-4178-b1c3-3fdf24524dea-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.330807 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330728    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/812cb630-c9f1-4b19-b5e2-b57fda496558-serving-cert\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.330807 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330757    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.330807 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330785    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.330914 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330806    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.330914 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330856    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.330914 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330877    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.331039 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330931    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-tmp\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.331039 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330958    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cj6q\" (UniqueName: \"kubernetes.io/projected/4b5dfc1c-8c00-45aa-b941-64f36a914db3-kube-api-access-2cj6q\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.331039 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.330997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hj5f\" (UniqueName: \"kubernetes.io/projected/71d83910-eb9d-4178-b1c3-3fdf24524dea-kube-api-access-9hj5f\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.331039 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331025    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.331181 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331043    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.331181 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331070    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.331181 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331114    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwfhk\" (UniqueName: \"kubernetes.io/projected/cc378d17-492d-4255-bd20-998d518e8b04-kube-api-access-kwfhk\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.331181 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331138    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.331181 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331158    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.331385 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331262    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjhgp\" (UniqueName: \"kubernetes.io/projected/f2f6f200-9701-4977-9256-68945f6568f2-kube-api-access-sjhgp\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.331385 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331299    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fq9b\" (UniqueName: \"kubernetes.io/projected/d91a8683-96c3-4725-b90b-da6590c5076e-kube-api-access-7fq9b\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.331385 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331354    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/81511c2e-f1cb-40ea-8d52-b795feb55690-config-volume\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.331526 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331386    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7sth\" (UniqueName: \"kubernetes.io/projected/fae5b32c-a10a-442a-8d19-e5ab9221ed37-kube-api-access-w7sth\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.331526 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331458    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xqks\" (UniqueName: \"kubernetes.io/projected/c6a49d28-717c-4fab-91be-c77a2e58b037-kube-api-access-7xqks\") pod \"volume-data-source-validator-58c9f5f46d-pnqc5\" (UID: \"c6a49d28-717c-4fab-91be-c77a2e58b037\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:11.331526 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331488    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.331656 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331522    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-trusted-ca\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.331656 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331525    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-snapshots\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.331656 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331574    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-service-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.331656 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331632    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b5dfc1c-8c00-45aa-b941-64f36a914db3-serving-cert\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.331831 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331659    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lg2q\" (UniqueName: \"kubernetes.io/projected/81511c2e-f1cb-40ea-8d52-b795feb55690-kube-api-access-7lg2q\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.331831 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331729    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91a8683-96c3-4725-b90b-da6590c5076e-serving-cert\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.331831 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331760    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.331831 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331795    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-config\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.331831 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331826    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.332119 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331845    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.332119 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331863    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.332119 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331893    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksq2c\" (UniqueName: \"kubernetes.io/projected/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d-kube-api-access-ksq2c\") pod \"kas-connection-checker-685895c7c9-rxq5k\" (UID: \"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d\") " pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:11.332119 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331953    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2f6f200-9701-4977-9256-68945f6568f2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.332119 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.331995    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b5dfc1c-8c00-45aa-b941-64f36a914db3-config\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.332119 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332041    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332122    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332150    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq227\" (UniqueName: \"kubernetes.io/projected/762e65a1-8e7f-4d15-945c-c29c442ac478-kube-api-access-zq227\") pod \"network-check-source-6f78d59f99-4gg4t\" (UID: \"762e65a1-8e7f-4d15-945c-c29c442ac478\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332177    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-stats-auth\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332209    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrvmm\" (UniqueName: \"kubernetes.io/projected/42fd8ec3-88cb-4e0b-9d58-1c654ff20582-kube-api-access-vrvmm\") pod \"kas-connection-checker-685895c7c9-lkzkr\" (UID: \"42fd8ec3-88cb-4e0b-9d58-1c654ff20582\") " pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332241    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc4xk\" (UniqueName: \"kubernetes.io/projected/514791bf-52ab-46d1-bec2-44baa1b3f48d-kube-api-access-kc4xk\") pod \"kas-connection-checker-685895c7c9-r6xqj\" (UID: \"514791bf-52ab-46d1-bec2-44baa1b3f48d\") " pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332265    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332282    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc378d17-492d-4255-bd20-998d518e8b04-cert\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332297    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332311    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332330    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332345    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2f6f200-9701-4977-9256-68945f6568f2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.332407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.332341    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-config\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.335048 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.335032    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/812cb630-c9f1-4b19-b5e2-b57fda496558-serving-cert\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.339011 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.338982    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jr28\" (UniqueName: \"kubernetes.io/projected/812cb630-c9f1-4b19-b5e2-b57fda496558-kube-api-access-4jr28\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.339393 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.339366    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksq2c\" (UniqueName: \"kubernetes.io/projected/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d-kube-api-access-ksq2c\") pod \"kas-connection-checker-685895c7c9-rxq5k\" (UID: \"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d\") " pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:11.339869 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.339846    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrvmm\" (UniqueName: \"kubernetes.io/projected/42fd8ec3-88cb-4e0b-9d58-1c654ff20582-kube-api-access-vrvmm\") pod \"kas-connection-checker-685895c7c9-lkzkr\" (UID: \"42fd8ec3-88cb-4e0b-9d58-1c654ff20582\") " pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:11.339991 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.339956    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc4xk\" (UniqueName: \"kubernetes.io/projected/514791bf-52ab-46d1-bec2-44baa1b3f48d-kube-api-access-kc4xk\") pod \"kas-connection-checker-685895c7c9-r6xqj\" (UID: \"514791bf-52ab-46d1-bec2-44baa1b3f48d\") " pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:11.433534 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433429    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.433534 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433472    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.433534 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433514    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.433817 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433550    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-tmp\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.433817 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433572    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cj6q\" (UniqueName: \"kubernetes.io/projected/4b5dfc1c-8c00-45aa-b941-64f36a914db3-kube-api-access-2cj6q\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.433817 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.433591    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:11.433817 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.433613    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:11.433817 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.433694    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.933672635 +0000 UTC m=+31.666502199 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:11.433817 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433595    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hj5f\" (UniqueName: \"kubernetes.io/projected/71d83910-eb9d-4178-b1c3-3fdf24524dea-kube-api-access-9hj5f\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433889    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433931    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433954    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433980    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfhk\" (UniqueName: \"kubernetes.io/projected/cc378d17-492d-4255-bd20-998d518e8b04-kube-api-access-kwfhk\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.433979    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-tmp\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434005    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434030    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434062    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjhgp\" (UniqueName: \"kubernetes.io/projected/f2f6f200-9701-4977-9256-68945f6568f2-kube-api-access-sjhgp\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.434086 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434087    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fq9b\" (UniqueName: \"kubernetes.io/projected/d91a8683-96c3-4725-b90b-da6590c5076e-kube-api-access-7fq9b\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434110    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/81511c2e-f1cb-40ea-8d52-b795feb55690-config-volume\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434140    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7sth\" (UniqueName: \"kubernetes.io/projected/fae5b32c-a10a-442a-8d19-e5ab9221ed37-kube-api-access-w7sth\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434170    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xqks\" (UniqueName: \"kubernetes.io/projected/c6a49d28-717c-4fab-91be-c77a2e58b037-kube-api-access-7xqks\") pod \"volume-data-source-validator-58c9f5f46d-pnqc5\" (UID: \"c6a49d28-717c-4fab-91be-c77a2e58b037\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434204    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434228    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-snapshots\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434256    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-service-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434286    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b5dfc1c-8c00-45aa-b941-64f36a914db3-serving-cert\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434309    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg2q\" (UniqueName: \"kubernetes.io/projected/81511c2e-f1cb-40ea-8d52-b795feb55690-kube-api-access-7lg2q\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434355    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91a8683-96c3-4725-b90b-da6590c5076e-serving-cert\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434381    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434419    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434447    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434517    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.434545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434537    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2f6f200-9701-4977-9256-68945f6568f2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434565    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b5dfc1c-8c00-45aa-b941-64f36a914db3-config\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434593    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434612    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434624    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434660    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq227\" (UniqueName: \"kubernetes.io/projected/762e65a1-8e7f-4d15-945c-c29c442ac478-kube-api-access-zq227\") pod \"network-check-source-6f78d59f99-4gg4t\" (UID: \"762e65a1-8e7f-4d15-945c-c29c442ac478\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434689    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-stats-auth\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434722    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434752    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc378d17-492d-4255-bd20-998d518e8b04-cert\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434776    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434800    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434831    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434856    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2f6f200-9701-4977-9256-68945f6568f2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434874    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434886    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhph\" (UniqueName: \"kubernetes.io/projected/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-kube-api-access-nqhph\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.434966    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9918792e-83ee-41de-8794-bf799e14788d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.435249 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.435029    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435076    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-service-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.435129    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert podName:9918792e-83ee-41de-8794-bf799e14788d nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.935113618 +0000 UTC m=+31.667943175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-nk6b8" (UID: "9918792e-83ee-41de-8794-bf799e14788d") : secret "networking-console-plugin-cert" not found
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435165    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-default-certificate\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435210    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81511c2e-f1cb-40ea-8d52-b795feb55690-tmp-dir\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435248    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435287    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435319    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435358    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/71d83910-eb9d-4178-b1c3-3fdf24524dea-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435390    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435413    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.435547    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.435593    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls podName:fae5b32c-a10a-442a-8d19-e5ab9221ed37 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.935578149 +0000 UTC m=+31.668407708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-v872x" (UID: "fae5b32c-a10a-442a-8d19-e5ab9221ed37") : secret "samples-operator-tls" not found
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435698    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9918792e-83ee-41de-8794-bf799e14788d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.436026 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.435879    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/81511c2e-f1cb-40ea-8d52-b795feb55690-config-volume\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.436688 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.436293    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81511c2e-f1cb-40ea-8d52-b795feb55690-tmp-dir\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.436688 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.436461    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 22:22:11.436688 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.436537    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls podName:81511c2e-f1cb-40ea-8d52-b795feb55690 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.936521269 +0000 UTC m=+31.669350827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls") pod "dns-default-fzbm2" (UID: "81511c2e-f1cb-40ea-8d52-b795feb55690") : secret "dns-default-metrics-tls" not found
Nov 01 22:22:11.436688 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.436581    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.436880 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.436691    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.936677036 +0000 UTC m=+31.669506614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:11.437230 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.437201    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.437520 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.437475    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.437622 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.437567    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-snapshots\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.437878 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.437857    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:11.437961 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.437912    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls podName:71d83910-eb9d-4178-b1c3-3fdf24524dea nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.937898576 +0000 UTC m=+31.670728138 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-l4759" (UID: "71d83910-eb9d-4178-b1c3-3fdf24524dea") : secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:11.438289 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.438264    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/71d83910-eb9d-4178-b1c3-3fdf24524dea-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.438376 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.438318    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.438623 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.438602    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc378d17-492d-4255-bd20-998d518e8b04-cert\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.438875 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.438854    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2f6f200-9701-4977-9256-68945f6568f2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.438965    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.439037    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.939012861 +0000 UTC m=+31.671842430 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : secret "router-metrics-certs-default" not found
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.438959    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.439048    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.439061    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.439105    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:11.939094625 +0000 UTC m=+31.671924204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439253    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439402    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91a8683-96c3-4725-b90b-da6590c5076e-serving-cert\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.439625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439589    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b5dfc1c-8c00-45aa-b941-64f36a914db3-config\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.440125 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439669    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.440125 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439773    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-default-certificate\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.440125 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439879    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b5dfc1c-8c00-45aa-b941-64f36a914db3-serving-cert\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.440125 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.439953    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.440342 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.440130    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.441460 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.441436    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-stats-auth\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.441647 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.441623    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2f6f200-9701-4977-9256-68945f6568f2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.443206 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.443179    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfhk\" (UniqueName: \"kubernetes.io/projected/cc378d17-492d-4255-bd20-998d518e8b04-kube-api-access-kwfhk\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.443591 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.443302    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cj6q\" (UniqueName: \"kubernetes.io/projected/4b5dfc1c-8c00-45aa-b941-64f36a914db3-kube-api-access-2cj6q\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.443686 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.443668    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hj5f\" (UniqueName: \"kubernetes.io/projected/71d83910-eb9d-4178-b1c3-3fdf24524dea-kube-api-access-9hj5f\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.444236 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.444091    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjhgp\" (UniqueName: \"kubernetes.io/projected/f2f6f200-9701-4977-9256-68945f6568f2-kube-api-access-sjhgp\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.444568 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.444544    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fq9b\" (UniqueName: \"kubernetes.io/projected/d91a8683-96c3-4725-b90b-da6590c5076e-kube-api-access-7fq9b\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.444891 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.444869    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq227\" (UniqueName: \"kubernetes.io/projected/762e65a1-8e7f-4d15-945c-c29c442ac478-kube-api-access-zq227\") pod \"network-check-source-6f78d59f99-4gg4t\" (UID: \"762e65a1-8e7f-4d15-945c-c29c442ac478\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:11.446561 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.446462    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.446687 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.446666    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.446817 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.446799    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xqks\" (UniqueName: \"kubernetes.io/projected/c6a49d28-717c-4fab-91be-c77a2e58b037-kube-api-access-7xqks\") pod \"volume-data-source-validator-58c9f5f46d-pnqc5\" (UID: \"c6a49d28-717c-4fab-91be-c77a2e58b037\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:11.447131 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.447109    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhph\" (UniqueName: \"kubernetes.io/projected/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-kube-api-access-nqhph\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.447816 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.447780    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7sth\" (UniqueName: \"kubernetes.io/projected/fae5b32c-a10a-442a-8d19-e5ab9221ed37-kube-api-access-w7sth\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.448061 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.448044    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.448343 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.448324    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.448609 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.448594    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg2q\" (UniqueName: \"kubernetes.io/projected/81511c2e-f1cb-40ea-8d52-b795feb55690-kube-api-access-7lg2q\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.466326 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.466307    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:11.487824 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.487802    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:11.490953 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.490933    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:11.505050 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.505025    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:11.516526 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.516305    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:11.529563 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.529165    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:11.542695 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.541701    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:11.544068 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.543994    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:11.577931 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.577039    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:11.582522 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.581759    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:11.757662 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.757631    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-rxq5k"]
Nov 01 22:22:11.760792 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.760767    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e3cf582_d2fe_4d46_8945_e8b1bfe6bf0d.slice/crio-a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a WatchSource:0}: Error finding container a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a: Status 404 returned error can't find the container with id a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a
Nov 01 22:22:11.773319 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.773297    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:11.773464 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.773445    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:11.773545 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.773481    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:11.777733 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.777713    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 22:22:11.777885 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.777815    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5scqq\""
Nov 01 22:22:11.777945 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.777915    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-fpxpg\""
Nov 01 22:22:11.777994 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.777952    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9pl6q\""
Nov 01 22:22:11.779292 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.779273    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 22:22:11.784390 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.784369    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-r6xqj"]
Nov 01 22:22:11.789012 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.788987    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod514791bf_52ab_46d1_bec2_44baa1b3f48d.slice/crio-64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752 WatchSource:0}: Error finding container 64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752: Status 404 returned error can't find the container with id 64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752
Nov 01 22:22:11.806731 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.806440    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lkzkr"]
Nov 01 22:22:11.822765 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.822739    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"]
Nov 01 22:22:11.825915 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.825891    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812cb630_c9f1_4b19_b5e2_b57fda496558.slice/crio-6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81 WatchSource:0}: Error finding container 6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81: Status 404 returned error can't find the container with id 6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81
Nov 01 22:22:11.855881 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.855856    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-fvxj8"]
Nov 01 22:22:11.856984 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.856957    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd91a8683_96c3_4725_b90b_da6590c5076e.slice/crio-9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9 WatchSource:0}: Error finding container 9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9: Status 404 returned error can't find the container with id 9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9
Nov 01 22:22:11.858788 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.858709    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"]
Nov 01 22:22:11.863069 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.862994    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:11.906943 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.906909    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"]
Nov 01 22:22:11.913191 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.913148    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5dfc1c_8c00_45aa_b941_64f36a914db3.slice/crio-5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151 WatchSource:0}: Error finding container 5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151: Status 404 returned error can't find the container with id 5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151
Nov 01 22:22:11.921200 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.921174    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"]
Nov 01 22:22:11.926540 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.926487    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6a49d28_717c_4fab_91be_c77a2e58b037.slice/crio-6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2 WatchSource:0}: Error finding container 6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2: Status 404 returned error can't find the container with id 6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2
Nov 01 22:22:11.937190 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.937170    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"]
Nov 01 22:22:11.939999 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.939931    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-j85jx"]
Nov 01 22:22:11.942022 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.941804    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:11.942022 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.941882    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:11.942022 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.941912    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:11.942022 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.941944    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:11.942022 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.941992    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:11.942273 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.942049    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:11.942273 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.942080    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.942273 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:11.942139    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:11.942411 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942273    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.942254023 +0000 UTC m=+32.675083597 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942612    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942674    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert podName:9918792e-83ee-41de-8794-bf799e14788d nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.942657335 +0000 UTC m=+32.675486912 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-nk6b8" (UID: "9918792e-83ee-41de-8794-bf799e14788d") : secret "networking-console-plugin-cert" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942677    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942729    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls podName:71d83910-eb9d-4178-b1c3-3fdf24524dea nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.942712158 +0000 UTC m=+32.675541735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-l4759" (UID: "71d83910-eb9d-4178-b1c3-3fdf24524dea") : secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942744    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942783    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls podName:81511c2e-f1cb-40ea-8d52-b795feb55690 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.942770602 +0000 UTC m=+32.675600173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls") pod "dns-default-fzbm2" (UID: "81511c2e-f1cb-40ea-8d52-b795feb55690") : secret "dns-default-metrics-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942794    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942829    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.942818597 +0000 UTC m=+32.675648181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : secret "router-metrics-certs-default" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942848    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942860    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942891    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.942880398 +0000 UTC m=+32.675709975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942947    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.942979    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls podName:fae5b32c-a10a-442a-8d19-e5ab9221ed37 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.9429694 +0000 UTC m=+32.675798985 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-v872x" (UID: "fae5b32c-a10a-442a-8d19-e5ab9221ed37") : secret "samples-operator-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.943041    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.943051    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:11.944409 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:11.943078    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:12.943068634 +0000 UTC m=+32.675898191 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:11.945241 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.943203    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod762e65a1_8e7f_4d15_945c_c29c442ac478.slice/crio-123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574 WatchSource:0}: Error finding container 123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574: Status 404 returned error can't find the container with id 123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574
Nov 01 22:22:11.950667 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:11.945551    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc378d17_492d_4255_bd20_998d518e8b04.slice/crio-b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed WatchSource:0}: Error finding container b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed: Status 404 returned error can't find the container with id b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed
Nov 01 22:22:12.016158 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.016127    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:12.041150 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.041124    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t" event={"ID":"762e65a1-8e7f-4d15-945c-c29c442ac478","Type":"ContainerStarted","Data":"123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574"}
Nov 01 22:22:12.042539 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.042510    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-r6xqj" event={"ID":"514791bf-52ab-46d1-bec2-44baa1b3f48d","Type":"ContainerStarted","Data":"9be2a0364f3ca9f2788c07c5868b0039c782420aa2eb15d0412567ed704fe2bb"}
Nov 01 22:22:12.042664 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.042557    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-r6xqj" event={"ID":"514791bf-52ab-46d1-bec2-44baa1b3f48d","Type":"ContainerStarted","Data":"64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752"}
Nov 01 22:22:12.043623 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.043602    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5" event={"ID":"c6a49d28-717c-4fab-91be-c77a2e58b037","Type":"ContainerStarted","Data":"6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2"}
Nov 01 22:22:12.044534 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.044514    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5" event={"ID":"f2f6f200-9701-4977-9256-68945f6568f2","Type":"ContainerStarted","Data":"54a8069f0fe82192a44a46f19e66d6e30bedb7db2edb572c0aa62bceb6ea8827"}
Nov 01 22:22:12.045607 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.045586    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" event={"ID":"812cb630-c9f1-4b19-b5e2-b57fda496558","Type":"ContainerStarted","Data":"6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81"}
Nov 01 22:22:12.047042 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.047012    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-lkzkr" event={"ID":"42fd8ec3-88cb-4e0b-9d58-1c654ff20582","Type":"ContainerStarted","Data":"938326f4c7a1e50cdd2aa1a68753d03dbeb6750f68bcdb1831022a7115f812b1"}
Nov 01 22:22:12.047128 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.047060    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-lkzkr" event={"ID":"42fd8ec3-88cb-4e0b-9d58-1c654ff20582","Type":"ContainerStarted","Data":"fd47b453a0add5faad8d65a4ac879df0d48ec3b9e2c9585603e5e2126be98ad1"}
Nov 01 22:22:12.048156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.048136    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8" event={"ID":"4b5dfc1c-8c00-45aa-b941-64f36a914db3","Type":"ContainerStarted","Data":"5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151"}
Nov 01 22:22:12.049224 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.049206    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" event={"ID":"d91a8683-96c3-4725-b90b-da6590c5076e","Type":"ContainerStarted","Data":"9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9"}
Nov 01 22:22:12.050596 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.050570    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-rxq5k" event={"ID":"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d","Type":"ContainerStarted","Data":"8463a6af65a0c79f651d4b65585411559a64a716922a84d8b89563ca2b32ee2e"}
Nov 01 22:22:12.050677 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.050600    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-rxq5k" event={"ID":"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d","Type":"ContainerStarted","Data":"a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a"}
Nov 01 22:22:12.051690 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.051673    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j85jx" event={"ID":"cc378d17-492d-4255-bd20-998d518e8b04","Type":"ContainerStarted","Data":"b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed"}
Nov 01 22:22:12.057260 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.057223    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-r6xqj" podStartSLOduration=426.057213259 podStartE2EDuration="7m6.057213259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:12.057009533 +0000 UTC m=+31.789839111" watchObservedRunningTime="2026-11-01 22:22:12.057213259 +0000 UTC m=+31.790042837"
Nov 01 22:22:12.072188 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.072150    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-lkzkr" podStartSLOduration=426.072142028 podStartE2EDuration="7m6.072142028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:12.071704611 +0000 UTC m=+31.804534191" watchObservedRunningTime="2026-11-01 22:22:12.072142028 +0000 UTC m=+31.804971607"
Nov 01 22:22:12.088860 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.088824    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-rxq5k" podStartSLOduration=426.0888139 podStartE2EDuration="7m6.0888139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:12.088011855 +0000 UTC m=+31.820841435" watchObservedRunningTime="2026-11-01 22:22:12.0888139 +0000 UTC m=+31.821643478"
Nov 01 22:22:12.175218 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.175194    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:12.952425 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952391    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:12.952625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952437    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:12.952625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952468    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:12.952625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952528    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:12.952625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952586    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:12.952625 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952613    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:12.952855 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952658    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:12.952855 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:12.952686    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:12.952855 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.952831    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 22:22:12.952972 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.952890    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls podName:81511c2e-f1cb-40ea-8d52-b795feb55690 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.952872498 +0000 UTC m=+34.685702072 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls") pod "dns-default-fzbm2" (UID: "81511c2e-f1cb-40ea-8d52-b795feb55690") : secret "dns-default-metrics-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953122    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953193    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert podName:9918792e-83ee-41de-8794-bf799e14788d nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.953162777 +0000 UTC m=+34.685992339 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-nk6b8" (UID: "9918792e-83ee-41de-8794-bf799e14788d") : secret "networking-console-plugin-cert" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953262    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953267    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953278    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953304    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls podName:71d83910-eb9d-4178-b1c3-3fdf24524dea nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.953292414 +0000 UTC m=+34.686121984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-l4759" (UID: "71d83910-eb9d-4178-b1c3-3fdf24524dea") : secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953321    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.953312133 +0000 UTC m=+34.686141696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953348    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953369    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953379    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls podName:fae5b32c-a10a-442a-8d19-e5ab9221ed37 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.953367977 +0000 UTC m=+34.686197545 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-v872x" (UID: "fae5b32c-a10a-442a-8d19-e5ab9221ed37") : secret "samples-operator-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953399    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.953390301 +0000 UTC m=+34.686219873 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : secret "router-metrics-certs-default" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953429    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953438    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953461    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.9534511 +0000 UTC m=+34.686280672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:12.953511 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:12.953476    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:14.95346864 +0000 UTC m=+34.686298201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:14.467946 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.467899    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:14.468535 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.468107    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:14.468535 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.468261    2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 22:22:14.468535 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.468323    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:22:46.46830298 +0000 UTC m=+66.201132550 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : secret "metrics-daemon-secret" not found
Nov 01 22:22:14.471340 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.471311    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:14.569446 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.569065    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:14.572175 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.572125    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:14.582046 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.581850    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:14.821207 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.821133    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:14.898905 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.898874    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9s2pz"]
Nov 01 22:22:14.905385 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:14.905354    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod304f876e_02df_45da_af55_62cae8468a50.slice/crio-1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043 WatchSource:0}: Error finding container 1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043: Status 404 returned error can't find the container with id 1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043
Nov 01 22:22:14.974295 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974256    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:14.974439 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974308    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:14.974439 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974343    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:14.974439 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974406    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:14.974595 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974452    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:14.974595 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974480    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:14.974595 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974539    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:14.974595 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:14.974566    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:14.974737 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.974696    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 22:22:14.974768 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.974738    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls podName:81511c2e-f1cb-40ea-8d52-b795feb55690 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.974725717 +0000 UTC m=+38.707555275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls") pod "dns-default-fzbm2" (UID: "81511c2e-f1cb-40ea-8d52-b795feb55690") : secret "dns-default-metrics-tls" not found
Nov 01 22:22:14.975159 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975142    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:14.975206 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975159    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:14.975247 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975205    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975195193 +0000 UTC m=+38.708024750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:14.975293 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975260    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 22:22:14.975329 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975294    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls podName:fae5b32c-a10a-442a-8d19-e5ab9221ed37 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975283914 +0000 UTC m=+38.708113483 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-v872x" (UID: "fae5b32c-a10a-442a-8d19-e5ab9221ed37") : secret "samples-operator-tls" not found
Nov 01 22:22:14.975373 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975360    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:14.975373 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975370    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:14.975450 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975397    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975387846 +0000 UTC m=+38.708217407 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:14.975523 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975474    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 22:22:14.975578 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975525    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert podName:9918792e-83ee-41de-8794-bf799e14788d nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975512684 +0000 UTC m=+38.708342245 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-nk6b8" (UID: "9918792e-83ee-41de-8794-bf799e14788d") : secret "networking-console-plugin-cert" not found
Nov 01 22:22:14.975637 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975580    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:14.975637 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975608    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls podName:71d83910-eb9d-4178-b1c3-3fdf24524dea nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975599178 +0000 UTC m=+38.708428737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-l4759" (UID: "71d83910-eb9d-4178-b1c3-3fdf24524dea") : secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:14.975723 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975658    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 22:22:14.975723 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975682    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975676639 +0000 UTC m=+38.708506195 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : secret "router-metrics-certs-default" not found
Nov 01 22:22:14.975822 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:14.975737    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:18.975728344 +0000 UTC m=+38.708557935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:15.070918 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:15.070473    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5" event={"ID":"c6a49d28-717c-4fab-91be-c77a2e58b037","Type":"ContainerStarted","Data":"912f997bad45b2e0b207a2471f25df2882a51f67dcef50ced0e203a2c45ecda5"}
Nov 01 22:22:15.071929 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:15.071867    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9s2pz" event={"ID":"304f876e-02df-45da-af55-62cae8468a50","Type":"ContainerStarted","Data":"1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043"}
Nov 01 22:22:15.085277 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:15.085227    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5" podStartSLOduration=396.028155193 podStartE2EDuration="6m38.08521484s" totalImagesPullingTime="2.057059647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:15.08441252 +0000 UTC m=+34.817242118" watchObservedRunningTime="2026-11-01 22:22:15.08521484 +0000 UTC m=+34.818044446"
Nov 01 22:22:15.129644 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:15.129618    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-lg7nw"]
Nov 01 22:22:15.132235 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:15.132206    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfbadae_7483_48c4_9fe4_a178c2e3c15b.slice/crio-8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90 WatchSource:0}: Error finding container 8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90: Status 404 returned error can't find the container with id 8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90
Nov 01 22:22:15.221309 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:15.221280    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:15.328950 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:15.328885    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:16.077183 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:16.077102    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lg7nw" event={"ID":"2bfbadae-7483-48c4-9fe4-a178c2e3c15b","Type":"ContainerStarted","Data":"8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90"}
Nov 01 22:22:16.081455 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:16.079972    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/0.log"
Nov 01 22:22:16.081455 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:16.080012    2584 generic.go:356] "Generic (PLEG): container finished" podID="812cb630-c9f1-4b19-b5e2-b57fda496558" containerID="c8dcc9708778c0b8b8cf10e7e890707b687ecdd509eea6f3cfad600c211ffdae" exitCode=255
Nov 01 22:22:16.081455 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:16.080760    2584 scope.go:122] "RemoveContainer" containerID="c8dcc9708778c0b8b8cf10e7e890707b687ecdd509eea6f3cfad600c211ffdae"
Nov 01 22:22:16.081455 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:16.081167    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" event={"ID":"812cb630-c9f1-4b19-b5e2-b57fda496558","Type":"ContainerDied","Data":"c8dcc9708778c0b8b8cf10e7e890707b687ecdd509eea6f3cfad600c211ffdae"}
Nov 01 22:22:17.090140 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:17.090104    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:22:17.091131 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:17.090817    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/0.log"
Nov 01 22:22:17.091131 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:17.090859    2584 generic.go:356] "Generic (PLEG): container finished" podID="812cb630-c9f1-4b19-b5e2-b57fda496558" containerID="e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d" exitCode=255
Nov 01 22:22:17.091131 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:17.090924    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" event={"ID":"812cb630-c9f1-4b19-b5e2-b57fda496558","Type":"ContainerDied","Data":"e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d"}
Nov 01 22:22:17.091131 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:17.090973    2584 scope.go:122] "RemoveContainer" containerID="c8dcc9708778c0b8b8cf10e7e890707b687ecdd509eea6f3cfad600c211ffdae"
Nov 01 22:22:17.091438 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:17.091209    2584 scope.go:122] "RemoveContainer" containerID="e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d"
Nov 01 22:22:17.091438 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:17.091409    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-zz5p7_openshift-console-operator(812cb630-c9f1-4b19-b5e2-b57fda496558)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" podUID="812cb630-c9f1-4b19-b5e2-b57fda496558"
Nov 01 22:22:18.102702 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.102661    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5" event={"ID":"f2f6f200-9701-4977-9256-68945f6568f2","Type":"ContainerStarted","Data":"9432b4a54378ce03e9c28e06e14985cd497dc49e5910706d8bb3f47cb50a538d"}
Nov 01 22:22:18.106229 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.106202    2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:22:18.106638 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.106608    2584 scope.go:122] "RemoveContainer" containerID="e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d"
Nov 01 22:22:18.108057 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:18.106885    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-zz5p7_openshift-console-operator(812cb630-c9f1-4b19-b5e2-b57fda496558)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" podUID="812cb630-c9f1-4b19-b5e2-b57fda496558"
Nov 01 22:22:18.111297 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.110861    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" event={"ID":"d91a8683-96c3-4725-b90b-da6590c5076e","Type":"ContainerStarted","Data":"4584d47bd57acc759d2c58db24b08b4fbb0e8061e6c64c16ba773dd0460a445b"}
Nov 01 22:22:18.150567 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.148376    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5" podStartSLOduration=389.127343397 podStartE2EDuration="6m35.148357643s" totalImagesPullingTime="6.021014246s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:18.12283556 +0000 UTC m=+37.855665143" watchObservedRunningTime="2026-11-01 22:22:18.148357643 +0000 UTC m=+37.881187223"
Nov 01 22:22:18.856707 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.856656    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" podStartSLOduration=395.841619342 podStartE2EDuration="6m41.856641265s" totalImagesPullingTime="6.015021923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:18.16683653 +0000 UTC m=+37.899666106" watchObservedRunningTime="2026-11-01 22:22:18.856641265 +0000 UTC m=+38.589470844"
Nov 01 22:22:18.857151 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.857133    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"]
Nov 01 22:22:18.874277 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.873221    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"]
Nov 01 22:22:18.875021 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.875000    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:18.886530 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.886309    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ffzxb\""
Nov 01 22:22:18.886530 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.886396    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:18.886530 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.886469    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 01 22:22:18.901670 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.901628    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"]
Nov 01 22:22:18.901798 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.901705    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"]
Nov 01 22:22:18.901868 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.901821    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:18.905988 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:18.905873    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:19.018407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018335    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:19.018407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018375    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4wsn\" (UniqueName: \"kubernetes.io/projected/7d5c4c29-2458-449a-9348-d36bef863863-kube-api-access-b4wsn\") pod \"migrator-66df5d496b-h4d74\" (UID: \"7d5c4c29-2458-449a-9348-d36bef863863\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:19.018407 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018400    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:19.018611 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018470    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4mbd\" (UniqueName: \"kubernetes.io/projected/567008ed-4e09-47b1-b76b-050e148b71d7-kube-api-access-v4mbd\") pod \"migrator-66df5d496b-nbpf2\" (UID: \"567008ed-4e09-47b1-b76b-050e148b71d7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:19.018611 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018520    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:19.018611 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018561    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:19.018704 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018640    2584 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 22:22:19.018704 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018643    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:19.018704 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018663    2584 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Nov 01 22:22:19.018704 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018682    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:19.018704 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018698    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls podName:81511c2e-f1cb-40ea-8d52-b795feb55690 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.018678916 +0000 UTC m=+46.751508478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls") pod "dns-default-fzbm2" (UID: "81511c2e-f1cb-40ea-8d52-b795feb55690") : secret "dns-default-metrics-tls" not found
Nov 01 22:22:19.018704 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018702    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018724    2584 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018735    2584 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018791    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls podName:fae5b32c-a10a-442a-8d19-e5ab9221ed37 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.01876173 +0000 UTC m=+46.751591301 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-v872x" (UID: "fae5b32c-a10a-442a-8d19-e5ab9221ed37") : secret "samples-operator-tls" not found
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018821    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.018809417 +0000 UTC m=+46.751638982 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018840    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.018831336 +0000 UTC m=+46.751660894 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.018864    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.018856043 +0000 UTC m=+46.751685614 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:19.018974 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.018915    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.019013    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.019041    2584 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.019058    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.019078    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert podName:9918792e-83ee-41de-8794-bf799e14788d nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.019064672 +0000 UTC m=+46.751894247 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-nk6b8" (UID: "9918792e-83ee-41de-8794-bf799e14788d") : secret "networking-console-plugin-cert" not found
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.019161    2584 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.019192    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls podName:71d83910-eb9d-4178-b1c3-3fdf24524dea nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.019184094 +0000 UTC m=+46.752013654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-l4759" (UID: "71d83910-eb9d-4178-b1c3-3fdf24524dea") : secret "cluster-monitoring-operator-tls" not found
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.019161    2584 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Nov 01 22:22:19.019285 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.019218    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.019212933 +0000 UTC m=+46.752042490 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : secret "router-metrics-certs-default" not found
Nov 01 22:22:19.030328 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.030231    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:19.102859 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.102825    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-f54gq"]
Nov 01 22:22:19.119712 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.119630    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wsn\" (UniqueName: \"kubernetes.io/projected/7d5c4c29-2458-449a-9348-d36bef863863-kube-api-access-b4wsn\") pod \"migrator-66df5d496b-h4d74\" (UID: \"7d5c4c29-2458-449a-9348-d36bef863863\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:19.119712 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.119701    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4mbd\" (UniqueName: \"kubernetes.io/projected/567008ed-4e09-47b1-b76b-050e148b71d7-kube-api-access-v4mbd\") pod \"migrator-66df5d496b-nbpf2\" (UID: \"567008ed-4e09-47b1-b76b-050e148b71d7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:19.133965 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.133888    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4wsn\" (UniqueName: \"kubernetes.io/projected/7d5c4c29-2458-449a-9348-d36bef863863-kube-api-access-b4wsn\") pod \"migrator-66df5d496b-h4d74\" (UID: \"7d5c4c29-2458-449a-9348-d36bef863863\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:19.135642 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.134594    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8" event={"ID":"4b5dfc1c-8c00-45aa-b941-64f36a914db3","Type":"ContainerStarted","Data":"dc4b9f883f249a6fdc085e9385c10b9146cf8cfa6c53d921793a206e81aec529"}
Nov 01 22:22:19.135642 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.134641    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-75nhs-z7sqb"]
Nov 01 22:22:19.135642 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.134700    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.136039 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.136017    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4mbd\" (UniqueName: \"kubernetes.io/projected/567008ed-4e09-47b1-b76b-050e148b71d7-kube-api-access-v4mbd\") pod \"migrator-66df5d496b-nbpf2\" (UID: \"567008ed-4e09-47b1-b76b-050e148b71d7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:19.138759 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.138712    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8" podStartSLOduration=395.803047201 podStartE2EDuration="6m42.13869716s" totalImagesPullingTime="6.335649959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:19.137077228 +0000 UTC m=+38.869906807" watchObservedRunningTime="2026-11-01 22:22:19.13869716 +0000 UTC m=+38.871526739"
Nov 01 22:22:19.141066 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.139999    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 01 22:22:19.141066 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.140076    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4bt99\""
Nov 01 22:22:19.153982 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.153956    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f54gq"]
Nov 01 22:22:19.154104 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.154014    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t" event={"ID":"762e65a1-8e7f-4d15-945c-c29c442ac478","Type":"ContainerStarted","Data":"73391ff8d4b55a80607403c154f8c6eebc4ef272212d4cb30883133fabc37e4c"}
Nov 01 22:22:19.154104 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.154045    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-75nhs-z7sqb"]
Nov 01 22:22:19.154301 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.154292    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.156208 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.156179    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 01 22:22:19.158318 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.158274    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t" podStartSLOduration=388.373117844 podStartE2EDuration="6m35.158258166s" totalImagesPullingTime="6.785140322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:19.156385586 +0000 UTC m=+38.889215166" watchObservedRunningTime="2026-11-01 22:22:19.158258166 +0000 UTC m=+38.891087745"
Nov 01 22:22:19.171897 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.171877    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dc544\""
Nov 01 22:22:19.185045 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.185023    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:19.218001 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.217977    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:19.236143 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.236102    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:19.300972 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.300852    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:19.321668 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.321368    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.321668 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.321406    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.321481    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb04c807-ee72-4833-a98b-689f42b18e4c-data-volume\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.321933    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.321997    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-szlnl\" (UniqueName: \"kubernetes.io/projected/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-api-access-szlnl\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.322035    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.322091    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb04c807-ee72-4833-a98b-689f42b18e4c-crio-socket\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.322113    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.323083 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.322201    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.409381 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.409343    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423454    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szlnl\" (UniqueName: \"kubernetes.io/projected/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-api-access-szlnl\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423527    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423590    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb04c807-ee72-4833-a98b-689f42b18e4c-crio-socket\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423614    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423662    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423715    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423731    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb04c807-ee72-4833-a98b-689f42b18e4c-crio-socket\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423740    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.423779    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423819    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb04c807-ee72-4833-a98b-689f42b18e4c-data-volume\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.423852    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:19.923829849 +0000 UTC m=+39.656659410 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.423890    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.423905    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.423943    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:19.923925534 +0000 UTC m=+39.656755098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : configmap "kube-rbac-proxy" not found
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.424106    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.424156 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.424106    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb04c807-ee72-4833-a98b-689f42b18e4c-data-volume\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.424947 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.424418    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.424947 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.424584    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.432209 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.432188    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.432574 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.432556    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szlnl\" (UniqueName: \"kubernetes.io/projected/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-api-access-szlnl\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.520173 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.520103    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:19.531949 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.531919    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"]
Nov 01 22:22:19.532760 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:19.532737    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod567008ed_4e09_47b1_b76b_050e148b71d7.slice/crio-cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1 WatchSource:0}: Error finding container cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1: Status 404 returned error can't find the container with id cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1
Nov 01 22:22:19.562286 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.562009    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"]
Nov 01 22:22:19.564122 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:19.564086    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d5c4c29_2458_449a_9348_d36bef863863.slice/crio-302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183 WatchSource:0}: Error finding container 302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183: Status 404 returned error can't find the container with id 302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183
Nov 01 22:22:19.574484 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.574451    2584 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-d47tp"]
Nov 01 22:22:19.608581 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.608352    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-d47tp"]
Nov 01 22:22:19.608581 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.608480    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.612950 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.612830    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v4m7g\""
Nov 01 22:22:19.612950 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.612883    2584 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 01 22:22:19.612950 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.612928    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 01 22:22:19.612950 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.612945    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 01 22:22:19.615788 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.613038    2584 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 01 22:22:19.727062 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.727018    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggmlj\" (UniqueName: \"kubernetes.io/projected/1dde0398-5d23-49ed-b073-ddf5a2984678-kube-api-access-ggmlj\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.727244 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.727193    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-key\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.727381 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.727350    2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-cabundle\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.818414 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.818327    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-75nhs-z7sqb"]
Nov 01 22:22:19.821648 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:19.821619    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12bf6db7_4db5_40b6_ae97_bf1f59bdc4a5.slice/crio-a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29 WatchSource:0}: Error finding container a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29: Status 404 returned error can't find the container with id a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29
Nov 01 22:22:19.828623 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.828596    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-cabundle\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.828727 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.828693    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggmlj\" (UniqueName: \"kubernetes.io/projected/1dde0398-5d23-49ed-b073-ddf5a2984678-kube-api-access-ggmlj\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.828802 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.828788    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-key\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.829856 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.829829    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-cabundle\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.831531 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.831508    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-key\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.840356 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.840332    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggmlj\" (UniqueName: \"kubernetes.io/projected/1dde0398-5d23-49ed-b073-ddf5a2984678-kube-api-access-ggmlj\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:19.931000 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.930965    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.931191 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.931162    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:19.931326 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.931311    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:19.931383 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.931372    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:20.931355776 +0000 UTC m=+40.664185350 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:19.931653 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.931550    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 01 22:22:19.931982 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:19.931934    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:20.931760289 +0000 UTC m=+40.664589851 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : configmap "kube-rbac-proxy" not found
Nov 01 22:22:19.932592 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.932565    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:19.938440 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:19.938264    2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:20.057547 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.057488    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:20.128413 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.128333    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2" event={"ID":"567008ed-4e09-47b1-b76b-050e148b71d7","Type":"ContainerStarted","Data":"cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1"}
Nov 01 22:22:20.130063 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.130037    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75nhs-z7sqb" event={"ID":"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5","Type":"ContainerStarted","Data":"f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553"}
Nov 01 22:22:20.130205 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.130082    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75nhs-z7sqb" event={"ID":"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5","Type":"ContainerStarted","Data":"a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29"}
Nov 01 22:22:20.131544 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.131517    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74" event={"ID":"7d5c4c29-2458-449a-9348-d36bef863863","Type":"ContainerStarted","Data":"302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183"}
Nov 01 22:22:20.133406 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.133355    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-j85jx" event={"ID":"cc378d17-492d-4255-bd20-998d518e8b04","Type":"ContainerStarted","Data":"7d348e40f0dd234938ae75979d1d089421cc0ee0f96ee6d54d847c3b7d9c92c2"}
Nov 01 22:22:20.135454 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.135426    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-lg7nw" event={"ID":"2bfbadae-7483-48c4-9fe4-a178c2e3c15b","Type":"ContainerStarted","Data":"863e1083b298d287217317645c958052854796651ecda331b207bdf3c9089db1"}
Nov 01 22:22:20.147932 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.147885    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-75nhs-z7sqb" podStartSLOduration=1.147873449 podStartE2EDuration="1.147873449s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:20.146817761 +0000 UTC m=+39.879647339" watchObservedRunningTime="2026-11-01 22:22:20.147873449 +0000 UTC m=+39.880703029"
Nov 01 22:22:20.166096 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.166042    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-lg7nw" podStartSLOduration=35.085625607 podStartE2EDuration="39.166025539s" totalImagesPullingTime="4.080399932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:20.16427734 +0000 UTC m=+39.897106930" watchObservedRunningTime="2026-11-01 22:22:20.166025539 +0000 UTC m=+39.898855119"
Nov 01 22:22:20.182725 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.182679    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-j85jx" podStartSLOduration=1.578424791 podStartE2EDuration="9.18266596s" totalImagesPullingTime="7.604241169s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:20.182089658 +0000 UTC m=+39.914919238" watchObservedRunningTime="2026-11-01 22:22:20.18266596 +0000 UTC m=+39.915495539"
Nov 01 22:22:20.234207 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.234173    2584 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-d47tp"]
Nov 01 22:22:20.239561 ip-10-0-141-148 kubenswrapper[2584]: W1101 22:22:20.239532    2584 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dde0398_5d23_49ed_b073_ddf5a2984678.slice/crio-bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837 WatchSource:0}: Error finding container bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837: Status 404 returned error can't find the container with id bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837
Nov 01 22:22:20.485348 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.485313    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:20.942833 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.942720    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:20.943059 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:20.942845    2584 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Nov 01 22:22:20.943059 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:20.942897    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:20.943059 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:20.942923    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:22.942895169 +0000 UTC m=+42.675724752 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : configmap "kube-rbac-proxy" not found
Nov 01 22:22:20.943059 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:20.942983    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:20.943059 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:20.943013    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:22.943005098 +0000 UTC m=+42.675834668 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:21.132675 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.132641    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:21.144366 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.144330    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp" event={"ID":"1dde0398-5d23-49ed-b073-ddf5a2984678","Type":"ContainerStarted","Data":"b0bd72e14716f98ea2d1e3d7ac9a884095503a6f7324e8719763ef78341efb3e"}
Nov 01 22:22:21.144566 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.144385    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp" event={"ID":"1dde0398-5d23-49ed-b073-ddf5a2984678","Type":"ContainerStarted","Data":"bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837"}
Nov 01 22:22:21.145152 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.145128    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:21.277080 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.277045    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:21.399439 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.399408    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:21.506123 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.506094    2584 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:21.506261 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.506153    2584 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:21.506543 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.506528    2584 scope.go:122] "RemoveContainer" containerID="e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d"
Nov 01 22:22:21.506744 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:21.506724    2584 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-zz5p7_openshift-console-operator(812cb630-c9f1-4b19-b5e2-b57fda496558)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" podUID="812cb630-c9f1-4b19-b5e2-b57fda496558"
Nov 01 22:22:21.588844 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:21.588771    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:22.008080 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.008040    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:22.148873 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.148789    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74" event={"ID":"7d5c4c29-2458-449a-9348-d36bef863863","Type":"ContainerStarted","Data":"be6b70404359af8b30b29df1c9b422d6b87682e3bd8fccc521b810db37734c6a"}
Nov 01 22:22:22.150831 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.150801    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2" event={"ID":"567008ed-4e09-47b1-b76b-050e148b71d7","Type":"ContainerStarted","Data":"2a0e3e1d1585d1c972b36f01401995a11510b498d03b310da710d888854f599b"}
Nov 01 22:22:22.165268 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.165210    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74" podStartSLOduration=1.9718949829999999 podStartE2EDuration="4.165193758s" totalImagesPullingTime="2.193298775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:22.164271537 +0000 UTC m=+41.897101118" watchObservedRunningTime="2026-11-01 22:22:22.165193758 +0000 UTC m=+41.898023340"
Nov 01 22:22:22.165445 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.165415    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp" podStartSLOduration=3.165405017 podStartE2EDuration="3.165405017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:21.161629361 +0000 UTC m=+40.894458943" watchObservedRunningTime="2026-11-01 22:22:22.165405017 +0000 UTC m=+41.898234598"
Nov 01 22:22:22.183951 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.183898    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2" podStartSLOduration=1.955489135 podStartE2EDuration="4.183881806s" totalImagesPullingTime="2.228392671s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:22.18283565 +0000 UTC m=+41.915665229" watchObservedRunningTime="2026-11-01 22:22:22.183881806 +0000 UTC m=+41.916711387"
Nov 01 22:22:22.789408 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.789380    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:22.963627 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.963585    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:22.963804 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.963787    2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:22.963905 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:22.963892    2584 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:22.963956 ip-10-0-141-148 kubenswrapper[2584]: E1101 22:22:22.963948    2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:26.963932883 +0000 UTC m=+46.696762444 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:22.964065 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.964045    2584 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:22.997066 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:22.997042    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:23.106981 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:23.106904    2584 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:23.156075 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:23.156044    2584 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9s2pz" event={"ID":"304f876e-02df-45da-af55-62cae8468a50","Type":"ContainerStarted","Data":"ecc694709390c3f6940a8cc013c71199753603982836be0844f092724f1ead40"}
Nov 01 22:22:23.171906 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:23.171853    2584 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9s2pz" podStartSLOduration=34.275672758 podStartE2EDuration="42.171840966s" totalImagesPullingTime="7.896168208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 22:21:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:23.170279498 +0000 UTC m=+42.903109078" watchObservedRunningTime="2026-11-01 22:22:23.171840966 +0000 UTC m=+42.904670545"
Nov 01 22:22:23.176668 ip-10-0-141-148 kubenswrapper[2584]: I1101 22:22:23.176645    2584 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 22:22:23.176836 ip-10-0-141-148 systemd[1]: Stopping Kubernetes Kubelet...
Nov 01 22:22:23.192957 ip-10-0-141-148 systemd[1]: kubelet.service: Deactivated successfully.
Nov 01 22:22:23.193122 ip-10-0-141-148 systemd[1]: Stopped Kubernetes Kubelet.
Nov 01 22:22:23.193170 ip-10-0-141-148 systemd[1]: kubelet.service: Consumed 3.181s CPU time.
Nov 01 22:22:23.204147 ip-10-0-141-148 systemd[1]: Starting Kubernetes Kubelet...
Nov 01 22:22:23.302137 ip-10-0-141-148 kubenswrapper[5057]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 22:22:23.302137 ip-10-0-141-148 kubenswrapper[5057]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Nov 01 22:22:23.302137 ip-10-0-141-148 kubenswrapper[5057]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 22:22:23.302137 ip-10-0-141-148 kubenswrapper[5057]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304192    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304297    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304306    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304311    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304315    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304321    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.304301 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304326    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304447    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304456    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304460    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304465    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304470    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304475    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304503    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304507    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304510    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304513    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304515    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304520    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304523    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304526    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304528    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304531    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304534    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304537    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.304729 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304539    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304542    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304544    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304547    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304550    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304552    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304556    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304558    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304561    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304563    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304567    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304569    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304572    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304584    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304587    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304591    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304593    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304596    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304599    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304602    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.305508 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304604    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304607    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304610    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304612    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304615    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304617    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304620    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304623    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304626    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304629    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304631    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304634    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304637    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304639    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304642    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304645    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304649    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304651    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304654    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.306305 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304658    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304661    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304663    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304666    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304671    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304681    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304686    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304694    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304699    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304704    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304707    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304710    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304713    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304715    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304718    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304721    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304723    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304726    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304729    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.307065 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304731    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304734    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304737    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304742    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304746    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304748    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304751    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304754    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304756    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304761    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304763    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304766    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304769    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304772    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304774    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304777    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304779    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304782    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304785    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304787    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.307824 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304790    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304798    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.304802    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305624    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305632    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305636    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305639    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305643    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305648    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305651    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305653    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305656    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305659    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305661    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305664    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305670    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305673    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305675    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305678    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.308646 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305681    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305689    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305692    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305694    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305697    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305700    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305702    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305705    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305709    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305713    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305716    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305719    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305722    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305725    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305729    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305738    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305741    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305744    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305747    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305749    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.309401 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305751    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305754    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305757    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305760    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305762    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305765    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305767    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305770    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305774    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305776    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305779    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305782    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305784    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305787    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305790    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305792    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305795    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305797    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305800    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305802    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.310224 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305806    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305808    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305811    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305813    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305816    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305818    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305821    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305823    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305831    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305834    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305837    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305839    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305842    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305844    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305847    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305850    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305852    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305855    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305857    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.311083 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305860    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305864    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305867    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305869    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305872    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305874    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305877    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305880    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305882    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305885    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305888    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305890    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305893    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305896    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305898    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305901    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305903    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305906    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305908    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305911    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.311860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305914    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305916    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305924    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305927    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305930    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305933    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305935    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305938    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305940    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305943    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.305945    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306530    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306537    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306540    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306543    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306546    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306549    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306551    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306554    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306556    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.312728 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306559    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306562    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306564    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306567    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306569    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306572    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306575    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306577    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306580    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306583    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306586    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306589    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306591    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306594    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306604    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306607    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306610    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306612    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306615    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306618    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.313543 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306620    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306622    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306625    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306628    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306631    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306633    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306636    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306638    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306640    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306643    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306645    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306648    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306650    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306653    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306656    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306659    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306667    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306670    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306672    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.314335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306675    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306678    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306680    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306683    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306688    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306690    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306693    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306695    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306703    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306706    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306708    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306711    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306713    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306717    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306720    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306724    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306727    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306730    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306734    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.315130 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306736    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306739    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306741    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306744    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306746    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306749    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306752    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306754    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306757    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306759    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306762    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306764    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306767    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306770    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306772    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306776    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306779    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306783    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306786    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.315901 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306788    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306791    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306793    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306801    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306804    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306806    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306809    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306811    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306814    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306816    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306819    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306821    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306823    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306826    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306828    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306835    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306838    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306841    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306844    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.306847    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306941    5057 flags.go:64] FLAG: --address="0.0.0.0"
Nov 01 22:22:23.316691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306954    5057 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306966    5057 flags.go:64] FLAG: --anonymous-auth="true"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306970    5057 flags.go:64] FLAG: --application-metrics-count-limit="100"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306975    5057 flags.go:64] FLAG: --authentication-token-webhook="false"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306978    5057 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306983    5057 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306991    5057 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306994    5057 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.306997    5057 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307003    5057 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307006    5057 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307009    5057 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307012    5057 flags.go:64] FLAG: --cgroup-root=""
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307015    5057 flags.go:64] FLAG: --cgroups-per-qos="true"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307018    5057 flags.go:64] FLAG: --client-ca-file=""
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307034    5057 flags.go:64] FLAG: --cloud-provider="external"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307039    5057 flags.go:64] FLAG: --cluster-dns="[]"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307046    5057 flags.go:64] FLAG: --cluster-domain=""
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307049    5057 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307052    5057 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307055    5057 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307059    5057 flags.go:64] FLAG: --container-log-max-files="5"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307063    5057 flags.go:64] FLAG: --container-log-max-size="10Mi"
Nov 01 22:22:23.317549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307066    5057 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307070    5057 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307073    5057 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307076    5057 flags.go:64] FLAG: --contention-profiling="false"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307079    5057 flags.go:64] FLAG: --cpu-cfs-quota="true"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307083    5057 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307087    5057 flags.go:64] FLAG: --cpu-manager-policy="none"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307090    5057 flags.go:64] FLAG: --cpu-manager-policy-options=""
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307094    5057 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307098    5057 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307101    5057 flags.go:64] FLAG: --enable-debugging-handlers="true"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307104    5057 flags.go:64] FLAG: --enable-load-reader="false"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307107    5057 flags.go:64] FLAG: --enable-server="true"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307110    5057 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307116    5057 flags.go:64] FLAG: --event-burst="100"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307119    5057 flags.go:64] FLAG: --event-qps="50"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307122    5057 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307125    5057 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307129    5057 flags.go:64] FLAG: --eviction-hard=""
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307134    5057 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307137    5057 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307140    5057 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307143    5057 flags.go:64] FLAG: --eviction-soft=""
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307146    5057 flags.go:64] FLAG: --eviction-soft-grace-period=""
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307149    5057 flags.go:64] FLAG: --exit-on-lock-contention="false"
Nov 01 22:22:23.318446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307157    5057 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307160    5057 flags.go:64] FLAG: --experimental-mounter-path=""
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307163    5057 flags.go:64] FLAG: --fail-cgroupv1="true"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307166    5057 flags.go:64] FLAG: --fail-swap-on="true"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307169    5057 flags.go:64] FLAG: --feature-gates=""
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307173    5057 flags.go:64] FLAG: --file-check-frequency="20s"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307176    5057 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307179    5057 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307183    5057 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307186    5057 flags.go:64] FLAG: --healthz-port="10248"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307189    5057 flags.go:64] FLAG: --help="false"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307192    5057 flags.go:64] FLAG: --hostname-override="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307196    5057 flags.go:64] FLAG: --housekeeping-interval="10s"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307199    5057 flags.go:64] FLAG: --http-check-frequency="20s"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307202    5057 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307206    5057 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307209    5057 flags.go:64] FLAG: --image-gc-high-threshold="85"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307212    5057 flags.go:64] FLAG: --image-gc-low-threshold="80"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307215    5057 flags.go:64] FLAG: --image-service-endpoint=""
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307218    5057 flags.go:64] FLAG: --kernel-memcg-notification="false"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307221    5057 flags.go:64] FLAG: --kube-api-burst="100"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307224    5057 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307227    5057 flags.go:64] FLAG: --kube-api-qps="50"
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307229    5057 flags.go:64] FLAG: --kube-reserved=""
Nov 01 22:22:23.319556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307232    5057 flags.go:64] FLAG: --kube-reserved-cgroup=""
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307236    5057 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307239    5057 flags.go:64] FLAG: --kubelet-cgroups=""
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307242    5057 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307245    5057 flags.go:64] FLAG: --lock-file=""
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307248    5057 flags.go:64] FLAG: --log-cadvisor-usage="false"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307251    5057 flags.go:64] FLAG: --log-flush-frequency="5s"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307253    5057 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307259    5057 flags.go:64] FLAG: --log-json-split-stream="false"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307268    5057 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307271    5057 flags.go:64] FLAG: --log-text-split-stream="false"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307274    5057 flags.go:64] FLAG: --logging-format="text"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307277    5057 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307280    5057 flags.go:64] FLAG: --make-iptables-util-chains="true"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307283    5057 flags.go:64] FLAG: --manifest-url=""
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307286    5057 flags.go:64] FLAG: --manifest-url-header=""
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307290    5057 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307293    5057 flags.go:64] FLAG: --max-open-files="1000000"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307297    5057 flags.go:64] FLAG: --max-pods="110"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307300    5057 flags.go:64] FLAG: --maximum-dead-containers="-1"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307304    5057 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307307    5057 flags.go:64] FLAG: --memory-manager-policy="None"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307310    5057 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307313    5057 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307316    5057 flags.go:64] FLAG: --node-ip="0.0.0.0"
Nov 01 22:22:23.320504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307319    5057 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307327    5057 flags.go:64] FLAG: --node-status-max-images="50"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307330    5057 flags.go:64] FLAG: --node-status-update-frequency="10s"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307333    5057 flags.go:64] FLAG: --oom-score-adj="-999"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307336    5057 flags.go:64] FLAG: --pod-cidr=""
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307339    5057 flags.go:64] FLAG: --pod-manifest-path=""
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307341    5057 flags.go:64] FLAG: --pod-max-pids="-1"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307344    5057 flags.go:64] FLAG: --pods-per-core="0"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307348    5057 flags.go:64] FLAG: --port="10250"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307351    5057 flags.go:64] FLAG: --protect-kernel-defaults="false"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307354    5057 flags.go:64] FLAG: --provider-id="aws:///us-east-2a/i-0fbe7c4503bda3241"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307357    5057 flags.go:64] FLAG: --qos-reserved=""
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307360    5057 flags.go:64] FLAG: --read-only-port="10255"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307363    5057 flags.go:64] FLAG: --register-node="true"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307366    5057 flags.go:64] FLAG: --register-with-taints=""
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307370    5057 flags.go:64] FLAG: --registry-burst="10"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307373    5057 flags.go:64] FLAG: --registry-qps="5"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307381    5057 flags.go:64] FLAG: --reserved-cpus=""
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307384    5057 flags.go:64] FLAG: --reserved-memory=""
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307388    5057 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307391    5057 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307394    5057 flags.go:64] FLAG: --rotate-certificates="false"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307397    5057 flags.go:64] FLAG: --rotate-server-certificates="false"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307400    5057 flags.go:64] FLAG: --runonce="false"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307403    5057 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307406    5057 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307409    5057 flags.go:64] FLAG: --seccomp-default="false"
Nov 01 22:22:23.321621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307414    5057 flags.go:64] FLAG: --serialize-image-pulls="true"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307417    5057 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307420    5057 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307423    5057 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307426    5057 flags.go:64] FLAG: --storage-driver-password="root"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307429    5057 flags.go:64] FLAG: --storage-driver-secure="false"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307432    5057 flags.go:64] FLAG: --storage-driver-table="stats"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307435    5057 flags.go:64] FLAG: --storage-driver-user="root"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307438    5057 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307441    5057 flags.go:64] FLAG: --sync-frequency="1m0s"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307444    5057 flags.go:64] FLAG: --system-cgroups=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307447    5057 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307452    5057 flags.go:64] FLAG: --system-reserved-cgroup=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307455    5057 flags.go:64] FLAG: --tls-cert-file=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307458    5057 flags.go:64] FLAG: --tls-cipher-suites="[]"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307466    5057 flags.go:64] FLAG: --tls-min-version=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307469    5057 flags.go:64] FLAG: --tls-private-key-file=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307472    5057 flags.go:64] FLAG: --topology-manager-policy="none"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307475    5057 flags.go:64] FLAG: --topology-manager-policy-options=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307478    5057 flags.go:64] FLAG: --topology-manager-scope="container"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307481    5057 flags.go:64] FLAG: --v="2"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307485    5057 flags.go:64] FLAG: --version="false"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307510    5057 flags.go:64] FLAG: --vmodule=""
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307521    5057 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.307524    5057 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Nov 01 22:22:23.322688 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308471    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308525    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308532    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308537    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308541    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308545    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308550    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308557    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308566    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308571    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308575    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308580    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308585    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308589    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308593    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308598    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308602    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308607    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308611    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.323691 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308615    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308626    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308630    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308634    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308638    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308642    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308647    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308652    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308656    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308661    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308665    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308669    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308697    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308707    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308711    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308715    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308720    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308724    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308729    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308733    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.324443 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308739    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308743    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308748    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308752    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308756    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308760    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308770    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308774    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308779    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308782    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308787    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308791    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308795    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308800    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308804    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308809    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308813    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308818    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308849    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308877    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.325234 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308881    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308885    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308888    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308894    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308899    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308903    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308907    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308910    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308914    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308917    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308920    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308927    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308930    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308932    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308935    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308938    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308941    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308944    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308946    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.326055 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308949    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308952    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308955    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308958    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308961    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308966    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308969    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308972    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308974    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308977    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308980    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308983    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308987    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308990    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308992    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308995    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.308998    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309001    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309008    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309012    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.326820 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309016    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309019    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309022    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309024    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309027    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309030    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309035    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309038    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.309083    5057 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309169    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309175    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309182    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309185    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309188    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309191    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309198    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.327696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309201    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309204    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309207    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309209    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309214    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309217    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309220    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309222    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309225    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309228    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309231    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309237    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309240    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309242    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309245    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309248    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309250    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309253    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309256    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309259    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.328327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309261    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309264    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309267    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309271    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309276    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309279    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309282    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309284    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309287    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309290    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309292    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309295    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309298    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309300    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309303    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309306    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309311    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309317    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309321    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.329144 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309324    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309327    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309329    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309333    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309340    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309343    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309345    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309348    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309351    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309354    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309359    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309361    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309364    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309367    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309369    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309372    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309376    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309379    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309381    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309384    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.329964 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309387    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309389    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309392    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309397    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309400    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309403    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309406    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309409    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309412    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309415    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309420    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309424    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309428    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309431    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309436    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309440    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309443    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309446    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309449    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.330813 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309452    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309454    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309457    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309460    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309462    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309465    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309468    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309471    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309476    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309481    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309484    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309487    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309503    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309507    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309514    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309517    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309520    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309523    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309526    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.331591 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309530    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.309536    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.313417    5057 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.313433    5057 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.313452    5057 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.313457    5057 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313525    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313530    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313534    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313537    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313540    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313543    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313546    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313549    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313552    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313555    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313558    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313561    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313564    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.332341 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313566    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313569    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313571    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313574    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313577    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313580    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313583    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313586    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313589    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313591    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313594    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313597    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313599    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313602    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313604    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313607    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313611    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313616    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313619    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.333124 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313621    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313625    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313628    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313631    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313634    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313637    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313639    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313642    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313645    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313648    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313650    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313654    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313656    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313659    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313661    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313664    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313667    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313671    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313675    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.333891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313678    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313681    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313684    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313687    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313689    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313692    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313695    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313697    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313700    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313703    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313705    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313708    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313710    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313713    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313715    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313718    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313720    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313723    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313725    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313728    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.334754 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313730    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313733    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313735    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313738    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313741    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313743    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313746    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313749    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313751    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313755    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313757    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313760    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313763    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313766    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313769    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313772    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313774    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313777    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313779    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313782    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.335588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313784    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313787    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313790    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313793    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313795    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313799    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313801    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313804    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313807    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313809    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313812    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313814    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313817    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313819    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.313822    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.313850    5057 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 22:22:23.336379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314029    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314035    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314040    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314043    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314046    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314049    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314052    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314054    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314057    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314060    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314062    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314065    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314068    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314071    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314073    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314076    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314078    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314081    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314084    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.336891 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314086    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314089    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314091    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314094    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314097    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314099    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314102    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314105    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314107    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314109    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314112    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314115    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314117    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314120    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314122    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314125    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314127    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314130    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314132    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314135    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.337542 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314137    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314140    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314142    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314145    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314148    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314150    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314153    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314155    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314158    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314160    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314163    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314166    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314168    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314171    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314173    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314175    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314178    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314181    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314183    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314186    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.338051 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314188    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314191    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314193    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314196    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314199    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314201    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314204    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314206    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314209    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314211    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314214    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314216    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314218    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314221    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314224    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314226    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314229    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314232    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314234    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.338621 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314237    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314240    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314242    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314245    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314247    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314250    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314252    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314255    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314257    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314260    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314263    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314266    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314269    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314273    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314276    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314278    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314281    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314283    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314286    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.339103 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314289    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314291    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314294    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314296    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314299    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314301    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314304    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314306    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314308    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.314334    5057 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314365    5057 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314369    5057 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314372    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314375    5057 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314378    5057 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314380    5057 feature_gate.go:386] unrecognized feature gate: EVPN
Nov 01 22:22:23.339666 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314383    5057 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314385    5057 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314388    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314390    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314393    5057 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314395    5057 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314398    5057 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314400    5057 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314403    5057 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314406    5057 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314408    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314410    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314413    5057 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314415    5057 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314418    5057 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314420    5057 feature_gate.go:386] unrecognized feature gate: SignatureStores
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314424    5057 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314427    5057 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314430    5057 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314432    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Nov 01 22:22:23.340050 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314435    5057 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314438    5057 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314440    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314443    5057 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314445    5057 feature_gate.go:386] unrecognized feature gate: Example
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314449    5057 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314452    5057 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314454    5057 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314457    5057 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314460    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314462    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314465    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314467    5057 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314470    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314472    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314475    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314477    5057 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314480    5057 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314482    5057 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Nov 01 22:22:23.340671 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314485    5057 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314488    5057 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314514    5057 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314518    5057 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314520    5057 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314523    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314525    5057 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314528    5057 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314530    5057 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314535    5057 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314538    5057 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314541    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314544    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314547    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314549    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314552    5057 feature_gate.go:386] unrecognized feature gate: Example2
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314554    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314557    5057 feature_gate.go:386] unrecognized feature gate: DualReplica
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314561    5057 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314564    5057 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Nov 01 22:22:23.341340 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314566    5057 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314569    5057 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314571    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314575    5057 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314577    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314580    5057 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314583    5057 feature_gate.go:386] unrecognized feature gate: NewOLM
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314585    5057 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314589    5057 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314592    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314596    5057 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314598    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314601    5057 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314603    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314606    5057 feature_gate.go:386] unrecognized feature gate: OSStreams
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314613    5057 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314616    5057 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314619    5057 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314621    5057 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Nov 01 22:22:23.341866 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314624    5057 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314626    5057 feature_gate.go:386] unrecognized feature gate: OVNObservability
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314630    5057 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314633    5057 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314636    5057 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314638    5057 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314642    5057 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314646    5057 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314649    5057 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314652    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314655    5057 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314659    5057 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314662    5057 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314664    5057 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314667    5057 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314670    5057 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314672    5057 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314675    5057 feature_gate.go:386] unrecognized feature gate: EventTTL
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314678    5057 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314681    5057 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Nov 01 22:22:23.342319 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314683    5057 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:23.314686    5057 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.314742    5057 server.go:957] "Client rotation is on, will bootstrap in background"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.315724    5057 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.315771    5057 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.315961    5057 server.go:1014] "Starting client certificate rotation"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.316137    5057 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.316185    5057 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-31 22:16:40 +0000 UTC" deadline="2028-06-23 23:01:22.758058295 +0000 UTC"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.316214    5057 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14400h38m59.441846026s"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.316456    5057 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.317592    5057 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.319060    5057 log.go:25] "Validated CRI v1 runtime API"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.320746    5057 log.go:25] "Validated CRI v1 image API"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.321372    5057 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Nov 01 22:22:23.342846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.323333    5057 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8f6eaa64-f5ec-4c28-ab6a-36ee7f609dac:/dev/nvme0n1p4 caa28fe3-5844-416b-951b-3672579012e1:/dev/nvme0n1p3]
Nov 01 22:22:23.343261 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.323352    5057 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574/userdata/shm major:0 minor:243 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043/userdata/shm major:0 minor:294 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183/userdata/shm major:0 minor:333 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/54a8069f0fe82192a44a46f19e66d6e30bedb7db2edb572c0aa62bceb6ea8827/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/54a8069f0fe82192a44a46f19e66d6e30bedb7db2edb572c0aa62bceb6ea8827/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151/userdata/shm major:0 minor:240 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81/userdata/shm major:0 minor:234 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/729125a8c1f724ad90522448c0cedd08e7c96f8d3480af183aff99252aee638b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/729125a8c1f724ad90522448c0cedd08e7c96f8d3480af183aff99252aee638b/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90/userdata/shm major:0 minor:296 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9/userdata/shm major:0 minor:236 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814/userdata/shm major:0 minor:47 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf/userdata/shm major:0 minor:91 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29/userdata/shm major:0 minor:342 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed/userdata/shm major:0 minor:245 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837/userdata/shm major:0 minor:358 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c242fd290495c5f75e5ee2419d953b8fe99310a4f3afa85260c023a03320050c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c242fd290495c5f75e5ee2419d953b8fe99310a4f3afa85260c023a03320050c/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1/userdata/shm major:0 minor:330 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fd47b453a0add5faad8d65a4ac879df0d48ec3b9e2c9585603e5e2126be98ad1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fd47b453a0add5faad8d65a4ac879df0d48ec3b9e2c9585603e5e2126be98ad1/userdata/shm major:0 minor:232 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~projected/kube-api-access-m9sgc:{mountpoint:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~projected/kube-api-access-m9sgc major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5/volumes/kubernetes.io~projected/kube-api-access-cznts:{mountpoint:/var/lib/kubelet/pods/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5/volumes/kubernetes.io~projected/kube-api-access-cznts major:0 minor:332 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1dde0398-5d23-49ed-b073-ddf5a2984678/volumes/kubernetes.io~projected/kube-api-access-ggmlj:{mountpoint:/var/lib/kubelet/pods/1dde0398-5d23-49ed-b073-ddf5a2984678/volumes/kubernetes.io~projected/kube-api-access-ggmlj major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1dde0398-5d23-49ed-b073-ddf5a2984678/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/1dde0398-5d23-49ed-b073-ddf5a2984678/volumes/kubernetes.io~secret/signing-key major:0 minor:354 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d/volumes/kubernetes.io~projected/kube-api-access-ksq2c:{mountpoint:/var/lib/kubelet/pods/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d/volumes/kubernetes.io~projected/kube-api-access-ksq2c major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/295d3d68-c2d1-497f-9a6b-7bd5017b2a41/volumes/kubernetes.io~projected/kube-api-access-swmmr:{mountpoint:/var/lib/kubelet/pods/295d3d68-c2d1-497f-9a6b-7bd5017b2a41/volumes/kubernetes.io~projected/kube-api-access-swmmr major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/298f62f9-b298-474c-adec-6b780cca9db8/volumes/kubernetes.io~projected/kube-api-access-nwpgl:{mountpoint:/var/lib/kubelet/pods/298f62f9-b298-474c-adec-6b780cca9db8/volumes/kubernetes.io~projected/kube-api-access-nwpgl major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2bfbadae-7483-48c4-9fe4-a178c2e3c15b/volumes/kubernetes.io~projected/kube-api-access-lqjkn:{mountpoint:/var/lib/kubelet/pods/2bfbadae-7483-48c4-9fe4-a178c2e3c15b/volumes/kubernetes.io~projected/kube-api-access-lqjkn major:0 minor:290 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/304f876e-02df-45da-af55-62cae8468a50/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/304f876e-02df-45da-af55-62cae8468a50/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:288 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42fd8ec3-88cb-4e0b-9d58-1c654ff20582/volumes/kubernetes.io~projected/kube-api-access-vrvmm:{mountpoint:/var/lib/kubelet/pods/42fd8ec3-88cb-4e0b-9d58-1c654ff20582/volumes/kubernetes.io~projected/kube-api-access-vrvmm major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b5dfc1c-8c00-45aa-b941-64f36a914db3/volumes/kubernetes.io~projected/kube-api-access-2cj6q:{mountpoint:/var/lib/kubelet/pods/4b5dfc1c-8c00-45aa-b941-64f36a914db3/volumes/kubernetes.io~projected/kube-api-access-2cj6q major:0 minor:215 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b5dfc1c-8c00-45aa-b941-64f36a914db3/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/4b5dfc1c-8c00-45aa-b941-64f36a914db3/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887/volumes/kubernetes.io~projected/kube-api-access-98blt:{mountpoint:/var/lib/kubelet/pods/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887/volumes/kubernetes.io~projected/kube-api-access-98blt major:0 minor:62 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/514791bf-52ab-46d1-bec2-44baa1b3f48d/volumes/kubernetes.io~projected/kube-api-access-kc4xk:{mountpoint:/var/lib/kubelet/pods/514791bf-52ab-46d1-bec2-44baa1b3f48d/volumes/kubernetes.io~projected/kube-api-access-kc4xk major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/567008ed-4e09-47b1-b76b-050e148b71d7/volumes/kubernetes.io~projected/kube-api-access-v4mbd:{mountpoint:/var/lib/kubelet/pods/567008ed-4e09-47b1-b76b-050e148b71d7/volumes/kubernetes.io~projected/kube-api-access-v4mbd major:0 minor:327 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/71d83910-eb9d-4178-b1c3-3fdf24524dea/volumes/kubernetes.io~projected/kube-api-access-9hj5f:{mountpoint:/var/lib/kubelet/pods/71d83910-eb9d-4178-b1c3-3fdf24524dea/volumes/kubernetes.io~projected/kube-api-access-9hj5f major:0 minor:216 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/762e65a1-8e7f-4d15-945c-c29c442ac478/volumes/kubernetes.io~projected/kube-api-access-zq227:{mountpoint:/var/lib/kubelet/pods/762e65a1-8e7f-4d15-945c-c29c442ac478/volumes/kubernetes.io~projected/kube-api-access-zq227 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7d5c4c29-2458-449a-9348-d36bef863863/volumes/kubernetes.io~projected/kube-api-access-b4wsn:{mountpoint:/var/lib/kubelet/pods/7d5c4c29-2458-449a-9348-d36bef863863/volumes/kubernetes.io~projected/kube-api-access-b4wsn major:0 minor:322 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/812cb630-c9f1-4b19-b5e2-b57fda496558/volumes/kubernetes.io~projected/kube-api-access-4jr28:{mountpoint:/var/lib/kubelet/pods/812cb630-c9f1-4b19-b5e2-b57fda496558/volumes/kubernetes.io~projected/kube-api-access-4jr28 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/812cb630-c9f1-4b19-b5e2-b57fda496558/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/812cb630-c9f1-4b19-b5e2-b57fda496558/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/81511c2e-f1cb-40ea-8d52-b795feb55690/volumes/kubernetes.io~projected/kube-api-access-7lg2q:{mountpoint:/var/lib/kubelet/pods/81511c2e-f1cb-40ea-8d52-b795feb55690/volumes/kubernetes.io~projected/kube-api-access-7lg2q major:0 minor:227 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~projected/kube-api-access-nqhph:{mountpoint:/var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~projected/kube-api-access-nqhph major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~secret/default-certificate major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~secret/stats-auth major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~projected/kube-api-access-w2sbv:{mountpoint:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~projected/kube-api-access-w2sbv major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b546a0cd-894e-46ab-b097-6684578a546c/volumes/kubernetes.io~projected/kube-api-access-ldv8d:{mountpoint:/var/lib/kubelet/pods/b546a0cd-894e-46ab-b097-6684578a546c/volumes/kubernetes.io~projected/kube-api-access-ldv8d major:0 minor:63 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~empty-dir/tmp major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~projected/kube-api-access-6fhbf:{mountpoint:/var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~projected/kube-api-access-6fhbf major:0 minor:64 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f/volumes/kubernetes.io~projected/kube-api-access-96spj:{mountpoint:/var/lib/kubelet/pods/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f/volumes/kubernetes.io~projected/kube-api-access-96spj major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c6a49d28-717c-4fab-91be-c77a2e58b037/volumes/kubernetes.io~projected/kube-api-access-7xqks:{mountpoint:/var/lib/kubelet/pods/c6a49d28-717c-4fab-91be-c77a2e58b037/volumes/kubernetes.io~projected/kube-api-access-7xqks major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c80d6c27-cb56-46c7-bed5-b96e23274d9c/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/c80d6c27-cb56-46c7-bed5-b96e23274d9c/volumes/kubernetes.io~secret/agent-certs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cb04c807-ee72-4833-a98b-689f42b18e4c/volumes/kubernetes.io~projected/kube-api-access-szlnl:{mountpoint:/var/lib/kubelet/pods/cb04c807-ee72-4833-a98b-689f42b18e4c/volumes/kubernetes.io~projected/kube-api-access-szlnl major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc378d17-492d-4255-bd20-998d518e8b04/volumes/kubernetes.io~projected/kube-api-access-kwfhk:{mountpoint:/var/lib/kubelet/pods/cc378d17-492d-4255-bd20-998d518e8b04/volumes/kubernetes.io~projected/kube-api-access-kwfhk major:0 minor:214 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc378d17-492d-4255-bd20-998d518e8b04/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/cc378d17-492d-4255-bd20-998d518e8b04/volumes/kubernetes.io~secret/cert major:0 minor:208 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc7341bb-795d-430a-a015-36004ec6723a/volumes/kubernetes.io~projected/kube-api-access-4njbt:{mountpoint:/var/lib/kubelet/pods/cc7341bb-795d-430a-a015-36004ec6723a/volumes/kubernetes.io~projected/kube-api-access-4njbt major:0 minor:66 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volumes/kubernetes.io~projected/kube-api-access-ms7s9:{mountpoint:/var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volumes/kubernetes.io~projected/kube-api-access-ms7s9 major:0 minor:67 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:65 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d91a8683-96c3-4725-b90b-da6590c5076e/volumes/kubernetes.io~projected/kube-api-access-7fq9b:{mountpoint:/var/lib/kubelet/pods/d91a8683-96c3-4725-b90b-da6590c5076e/volumes/kubernetes.io~projected/kube-api-access-7fq9b major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/d91a8683-96c3-4725-b90b-da6590c5076e/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/d91a8683-96c3-4725-b90b-da6590c5076e/volumes/kubernetes.io~secret/serving-cert major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f2f6f200-9701-4977-9256-68945f6568f2/volumes/kubernetes.io~projected/kube-api-access-sjhgp:{mountpoint:/var/lib/kubelet/pods/f2f6f200-9701-4977-9256-68945f6568f2/volumes/kubernetes.io~projected/kube-api-access-sjhgp major:0 minor:217 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f2f6f200-9701-4977-9256-68945f6568f2/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/f2f6f200-9701-4977-9256-68945f6568f2/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fae5b32c-a10a-442a-8d19-e5ab9221ed37/volumes/kubernetes.io~projected/kube-api-access-w7sth:{mountpoint:/var/lib/kubelet/pods/fae5b32c-a10a-442a-8d19-e5ab9221ed37/volumes/kubernetes.io~projected/kube-api-access-w7sth major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2/volumes/kubernetes.io~projected/kube-api-access-2jstc:{mountpoint:/var/lib/kubelet/pods/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2/volumes/kubernetes.io~projected/kube-api-access-2jstc major:0 minor:58 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/deeeca913d74673b6c9f6698e1175a71964d2603306fd2d8b5bb63f9a8cf6f79/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/624f1395178a7fa65172b246cb6f61c89b72783514c59f238aeb744ad50a2fac/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/e28884f1f8237e0566cd02d9c3ffcf13d22ab2c6591f847482ed3757f8da150d/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/4083a5548e5650bcd75c06b10c4a6e750762a4ade6c1d0ce30cd748127a3bac9/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-113:{mountpoint:/var/lib/containers/storage/overlay/bb7628853020b58ba691cba44314dfbc43b814993fa51331c708b45f28c78734/merged major:0 minor:113 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8228ee0bf07452cd1478a3897acd22262291e8d4587e4ce6e87129e85041f856/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-121:{mountpoint:/var/lib/containers/storage/overlay/735f900240eb41d69b3013300f30406539e37630d17cbae38fbde081828a0c16/merged major:0 minor:121 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/888425500b406073e31c9e5dcb88ade8899b254c88cb41605b11a7036fe51c09/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/9a920aaea4e89ae28166b852a1511808dc4ca02e5c9a7802c0f333a0695e7db2/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/6b14153b2d41006c190460fb358366403f2d352eb49d1ae292d03aff94a893d1/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/8e527e3fbc5b29060348f7959daaf10e09c28d15482ab62823dca9ce20540c3f/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/e2390a3d19163eff673d84058d9ebbd30e6bc4ca5235e3a6f6d04398475bacb1/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-149:{mountpoint:/var/lib/containers/storage/overlay/8fbc94f23b9aa07d308bec992ac05b11460de4460d41cbc689976766308f8d9e/merged major:0 minor:149 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/ad44273335be45d9e4ee04f3eb53a4818ccfec33cb10b11a9cf08872d05566f5/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/e3a657261b3ffeb9335d3c7c9738c5eb56986f1b3dbaa745c0f84a9a8b9f4746/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/e8e7742eae821a8f5c18da51bb18367c82284e65b4a2c3518ce8d8e266f77996/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/03b720f9a0a063d74cc231aa7722c6a5c043ca7f8fe993d3707e08770c2fcb7d/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/0637e235012d2f7de63897ec26f3fc2aca86c9298b74598cb2d394fb0d792a63/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/5495c8af0d1d83e00b9a2b3cae44507b3ea83a7623e8955881fc195348e466a7/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/981c53f0f088bdb54632c44a716225c227d9d7fd299df5219eb19b0c4338e89d/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/2083f56b6060f36d3fe86a1af51507a5e392d1769b812ba36a0b816e0319e011/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/f28889ff032d29b2250c7e1fd113d050b41c5863c9ef1411ecdef45f0dc38b52/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/83d91d03ada5502932013a7d13d44da5450d554a1d667bfd4b123cb5fa7d3d33/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/6889fbca380926b849ce38b9a7d4c227d573f6af73810c0f42f4aba4a3261ec3/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/f34da8e097ce6e82c461d278a49c8c40e2327cf3d580a66531dba98b24302654/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/90a1d5f6e5c80dcdd6e4a16bf4962fe4024a3e18d1bac7183b0c87f482d36c38/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/c06af84652d213343620218cc439122f782bb093501e5bf248800e8d57797b44/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-263:{mountpoint:/var/lib/containers/storage/overlay/5ab988c02d58f6ed7bcda84bc688624426fd72023c9402c2590b0964a6f9b08d/merged major:0 minor:263 fsType:overlay blockSize:0} overlay_0-265:{mountpoint:/var/lib/containers/storage/overlay/a31782fcb667b6abba472881b7e048ac8f045dfeee0de0c75e3fac4d699c7430/merged major:0 minor:265 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/3dde7fe841e1f92d95f8654c9047725af984658ae668d5df8d95ddd066045173/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/b778d8faefc9b5157fb9c0f1428bc5117571b47dea19325b9c4093130fc8092f/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/9b9e102b2626383e109a36ce04558ae9a64e7a037bdb0d3a9895a9dc892236d1/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-274:{mountpoint:/var/lib/containers/storage/overlay/681dc0d6d198815832aef27cdc2e905529587188e407b29092f64ba591dfaf62/merged major:0 minor:274 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/c34dd547637530e6cfe583a4992ac6edb73a05d8870fb8260fa9a7fb11b34311/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-278:{mountpoint:/var/lib/containers/storage/overlay/92e8c9e909330e3ad4073c7c97bc45640310e07f2d917faaa8ccfbaec2868c3e/merged major:0 minor:278 fsType:overlay blockSize:0} overlay_0-298:{mountpoint:/var/lib/containers/storage/overlay/dd8e9c151acc4099be8333a97e81bc5c4939e7a33801ebbb68758ae9d546ac07/merged major:0 minor:298 fsType:overlay blockSize:0} overlay_0-300:{mountpoint:/var/lib/containers/storage/overlay/3f119fb5025c3a5bcc37ebff240f2505b7091aa9d50dd056e9493bdb5d36b19d/merged major:0 minor:300 fsType:overlay blockSize:0} overlay_0-302:{mountpoint:/var/lib/containers/storage/overlay/55e49bb7bdaf4bee76b4349d4a24f3a9039360ab07f1250c358095409b383bd2/merged major:0 minor:302 fsType:overlay blockSize:0} overlay_0-304:{mountpoint:/var/lib/containers/storage/overlay/778cf9c5081b9f9a54eeebbd2fd1bb1ac9662274447e0576837e1404a0d2067e/merged major:0 minor:304 fsType:overlay blockSize:0} overlay_0-306:{mountpoint:/var/lib/containers/storage/overlay/da72b3d575a5e146c66b35c10f80bc7be5231089ab4f802dd45e1ea630e98c42/merged major:0 minor:306 fsType:overlay blockSize:0} overlay_0-316:{mountpoint:/var/lib/containers/storage/overlay/862143b248e253d7e72c00765dbbe55feca03953c1e836bec777ea7967a4b217/merged major:0 minor:316 fsType:overlay blockSize:0} overlay_0-328:{mountpoint:/var/lib/containers/storage/overlay/c9c9688a61e8379e6dcde9a01d9af9cce87974c8f56d0b2a04104026dd2a2a42/merged major:0 minor:328 fsType:overlay blockSize:0} overlay_0-340:{mountpoint:/var/lib/containers/storage/overlay/2a882af2df8560b54ccf9b64f1472a8c647bed43fb99a531e0f32bed8bfb00be/merged major:0 minor:340 fsType:overlay blockSize:0} overlay_0-344:{mountpoint:/var/lib/containers/storage/overlay/c71917ac9656c1beb6ec407923f80ea0eacd9d610c6462344580f8aea337fd06/merged major:0 minor:344 fsType:overlay blockSize:0} overlay_0-346:{mountpoint:/var/lib/containers/storage/overlay/b370a40cf2c92a352ec5b35667712617c20713cfc5256152ca3f134330daa1b4/merged major:0 minor:346 fsType:overlay blockSize:0} overlay_0-348:{mountpoint:/var/lib/containers/storage/overlay/4c5ac978459c5162d464882ea2d0c6bde1b7203c55b64f41f9318654b92d8f73/merged major:0 minor:348 fsType:overlay blockSize:0} overlay_0-356:{mountpoint:/var/lib/containers/storage/overlay/e477f21829b64b15dfc4f66559b7ced801b5826214c0e3c7407f5a6687e9301e/merged major:0 minor:356 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/0f45a529c3a798800ee0a8e607764f2584cc74bb4ec1225ada13773b44e53e4f/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-366:{mountpoint:/var/lib/containers/storage/overlay/fa808bc53d6c2877994200fcf286c838b1d31ec6f0c637d1bff1ed2fa40829ac/merged major:0 minor:366 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/45955788c2e35b7d44645d0dc9c341af97acf1d2b364f3e0ad46539958385092/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-374:{mountpoint:/var/lib/containers/storage/overlay/08cefc4a007446cf4f3590596c9f3c97d0dfbda73d2dac8c1772cc3a23e01ac7/merged major:0 minor:374 fsType:overlay blockSize:0} overlay_0-376:{mountpoint:/var/lib/containers/storage/overlay/5a65bd80d2a5bb4cdea5b52d7e1ad15ab398f422a1177bacf8699ab9859ebed7/merged major:0 minor:376 fsType:overlay blockSize:0} overlay_0-49:{mountpoint:/var/lib/containers/storage/overlay/a5877fcd46c2b3d670bb71b2baaa367a88c7a72b4ce114be6c09e23b5d11754a/merged major:0 minor:49 fsType:overlay blockSize:0} overlay_0-53:{mountpoint:/var/lib/containers/storage/overlay/941feb3f0ec8dbb26c43c4e35848a12b77bd747848f6092cf9b7ec882493e730/merged major:0 minor:53 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/96527e03b1f1398a6815849c33ece9f4949dc0a88340d425c874afde63a44b67/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/6510c24f8ec8b628dd4d710406269f36f498509f6a6d548743d3e9fb32239628/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/019f11f08e217bb98f8ad2138e1c864a1dd83fa8e11e0ecd91f617521aa625ac/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/a121895c96af2b16d38cb243c0c9ce90b9a55b555b579055fd6b114030001d37/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/03c56c25f879fdfb88c9c94aba83e4ad1cc4197e6eb4096927fe226a19362d08/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-89:{mountpoint:/var/lib/containers/storage/overlay/7f8d638de0f6d041a055fae7fa1bfeaa5ee8cdbc0b9eea2f4476048f77b311c8/merged major:0 minor:89 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/b9b72a03902ebba7b43f426a69393da1aa816b62df3807565f73e2db5d3a178c/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/4d3e761ab26dda842454ad5773e25be88210dbf81bb3bcc26f0828092b78e7f4/merged major:0 minor:96 fsType:overlay blockSize:0}]
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.332560    5057 manager.go:217] Machine: {Timestamp:2026-11-01 22:22:23.332088509 +0000 UTC m=+0.073784119 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100115 MemoryCapacity:33164328960 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26e013ac02e5adc7f556572b76b41b SystemUUID:ec26e013-ac02-e5ad-c7f5-56572b76b41b BootID:c6fb49fe-bb80-4333-9865-07fa678b8f26 Filesystems:[{Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-348 DeviceMajor:0 DeviceMinor:348 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f/volumes/kubernetes.io~projected/kube-api-access-96spj DeviceMajor:0 DeviceMinor:61 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d/volumes/kubernetes.io~projected/kube-api-access-ksq2c DeviceMajor:0 DeviceMinor:201 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-53 DeviceMajor:0 DeviceMinor:53 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b546a0cd-894e-46ab-b097-6684578a546c/volumes/kubernetes.io~projected/kube-api-access-ldv8d DeviceMajor:0 DeviceMinor:63 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/cc7341bb-795d-430a-a015-36004ec6723a/volumes/kubernetes.io~projected/kube-api-access-4njbt DeviceMajor:0 DeviceMinor:66 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/098bd7fdddd0c9b5c4386b730036231bf203f1b8b9da491f528ad6b5529855c9/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~projected/kube-api-access-nqhph DeviceMajor:0 DeviceMinor:223 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-263 DeviceMajor:0 DeviceMinor:263 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2/volumes/kubernetes.io~projected/kube-api-access-2jstc DeviceMajor:0 DeviceMinor:58 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6c23806e9343247143d781c80a96215ae9c2a7552b0a60aa78cc05a08ed63eea/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/d91a8683-96c3-4725-b90b-da6590c5076e/volumes/kubernetes.io~projected/kube-api-access-7fq9b DeviceMajor:0 DeviceMinor:218 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~projected/kube-api-access-6fhbf DeviceMajor:0 DeviceMinor:64 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887/volumes/kubernetes.io~projected/kube-api-access-98blt DeviceMajor:0 DeviceMinor:62 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b5dfc1c-8c00-45aa-b941-64f36a914db3/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fd47b453a0add5faad8d65a4ac879df0d48ec3b9e2c9585603e5e2126be98ad1/userdata/shm DeviceMajor:0 DeviceMinor:232 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-356 DeviceMajor:0 DeviceMinor:356 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c216e4106e1e455a5fff3565295b616de758342ad3eae2b5bea8f24e0aab50eb/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f2f6f200-9701-4977-9256-68945f6568f2/volumes/kubernetes.io~projected/kube-api-access-sjhgp DeviceMajor:0 DeviceMinor:217 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c80d6c27-cb56-46c7-bed5-b96e23274d9c/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:55 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-121 DeviceMajor:0 DeviceMinor:121 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-344 DeviceMajor:0 DeviceMinor:344 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/298f62f9-b298-474c-adec-6b780cca9db8/volumes/kubernetes.io~projected/kube-api-access-nwpgl DeviceMajor:0 DeviceMinor:59 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/514791bf-52ab-46d1-bec2-44baa1b3f48d/volumes/kubernetes.io~projected/kube-api-access-kc4xk DeviceMajor:0 DeviceMinor:203 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/64628b709fe46d4cd5a742c55bb6170bfce0b3bf788617913a9b7b4fb98dd752/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-328 DeviceMajor:0 DeviceMinor:328 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-149 DeviceMajor:0 DeviceMinor:149 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:204 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a9d4979cabc5462c23b8edce9d5135d0a9f8842f7a96662012ccf7b2a6d6c59a/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-274 DeviceMajor:0 DeviceMinor:274 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/d91a8683-96c3-4725-b90b-da6590c5076e/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:209 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/f2f6f200-9701-4977-9256-68945f6568f2/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9572f83e8b46a4617e267ee098f1fa8aa933d0500bd2462786a48d4730acaef9/userdata/shm DeviceMajor:0 DeviceMinor:236 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-300 DeviceMajor:0 DeviceMinor:300 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-346 DeviceMajor:0 DeviceMinor:346 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/81511c2e-f1cb-40ea-8d52-b795feb55690/volumes/kubernetes.io~projected/kube-api-access-7lg2q DeviceMajor:0 DeviceMinor:227 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-340 DeviceMajor:0 DeviceMinor:340 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-366 DeviceMajor:0 DeviceMinor:366 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/71d83910-eb9d-4178-b1c3-3fdf24524dea/volumes/kubernetes.io~projected/kube-api-access-9hj5f DeviceMajor:0 DeviceMinor:216 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bbb1573e4f4f87d3a862f6a91849d8fc0595fab0f6cef60f564af63651b26837/userdata/shm DeviceMajor:0 DeviceMinor:358 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c55163d64234e2987677ebf89f4786b0420bd6328800f9892040d1c8ec1080e3/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cc378d17-492d-4255-bd20-998d518e8b04/volumes/kubernetes.io~projected/kube-api-access-kwfhk DeviceMajor:0 DeviceMinor:214 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/762e65a1-8e7f-4d15-945c-c29c442ac478/volumes/kubernetes.io~projected/kube-api-access-zq227 DeviceMajor:0 DeviceMinor:219 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-278 DeviceMajor:0 DeviceMinor:278 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-306 DeviceMajor:0 DeviceMinor:306 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cb04c807-ee72-4833-a98b-689f42b18e4c/volumes/kubernetes.io~projected/kube-api-access-szlnl DeviceMajor:0 DeviceMinor:339 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:65 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~projected/kube-api-access-m9sgc DeviceMajor:0 DeviceMinor:225 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6344759d70c3f54f0b8d1478bcf97ebf5624a03022b6cea38ebee0f920b32b81/userdata/shm DeviceMajor:0 DeviceMinor:234 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29/userdata/shm DeviceMajor:0 DeviceMinor:342 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-376 DeviceMajor:0 DeviceMinor:376 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/54a8069f0fe82192a44a46f19e66d6e30bedb7db2edb572c0aa62bceb6ea8827/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5/volumes/kubernetes.io~projected/kube-api-access-cznts DeviceMajor:0 DeviceMinor:332 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-374 DeviceMajor:0 DeviceMinor:374 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:211 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-304 DeviceMajor:0 DeviceMinor:304 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7d5c4c29-2458-449a-9348-d36bef863863/volumes/kubernetes.io~projected/kube-api-access-b4wsn DeviceMajor:0 DeviceMinor:322 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/fae5b32c-a10a-442a-8d19-e5ab9221ed37/volumes/kubernetes.io~projected/kube-api-access-w7sth DeviceMajor:0 DeviceMinor:224 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b34ee95f4a012a08dc73b333b020f0a2826c68c5dddb095e53cb6fdae72387ed/userdata/shm DeviceMajor:0 DeviceMinor:245 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a1d42be8ea6cc8ed802af2c1089e47a2176dfd55e5c40076507f9bb17f05dadf/userdata/shm DeviceMajor:0 DeviceMinor:91 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632865792 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b5dfc1c-8c00-45aa-b941-64f36a914db3/volumes/kubernetes.io~projected/kube-api-access-2cj6q DeviceMajor:0 DeviceMinor:215 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1dde0398-5d23-49ed-b073-ddf5a2984678/volumes/kubernetes.io~projected/kube-api-access-ggmlj DeviceMajor:0 DeviceMinor:355 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cf01e723-91f6-4c22-8c8a-78b1cdd730af/volumes/kubernetes.io~projected/kube-api-access-ms7s9 DeviceMajor:0 DeviceMinor:67 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:206 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6d7588ae89226c0f44c2e824e4cf25c1a776668612aa53b80c75adf49af849e2/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d1018c05abc585d4f936416781d918c0cfae198d4e4a15ffb1e1e6ed8be7043/userdata/shm DeviceMajor:0 DeviceMinor:294 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-298 DeviceMajor:0 DeviceMinor:298 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:57 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-89 DeviceMajor:0 DeviceMinor:89 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/812cb630-c9f1-4b19-b5e2-b57fda496558/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:205 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2bfbadae-7483-48c4-9fe4-a178c2e3c15b/volumes/kubernetes.io~projected/kube-api-access-lqjkn DeviceMajor:0 DeviceMinor:290 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/302f89b9f3c38df3e73f1d52a64ce7135be242140b843b89a420dd8c21529183/userdata/shm DeviceMajor:0 DeviceMinor:333 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-49 DeviceMajor:0 DeviceMinor:49 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:221 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:226 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5a2f0d1d4c858eca90a6562733b9b1f3ee352fe864a0098a883b2db5f96f5151/userdata/shm DeviceMajor:0 DeviceMinor:240 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/c242fd290495c5f75e5ee2419d953b8fe99310a4f3afa85260c023a03320050c/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/812cb630-c9f1-4b19-b5e2-b57fda496558/volumes/kubernetes.io~projected/kube-api-access-4jr28 DeviceMajor:0 DeviceMinor:197 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:212 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/123fa29e13b0235c045518ffaeba3e803aad66b0b0f1a4bfbd23541d31722574/userdata/shm DeviceMajor:0 DeviceMinor:243 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-316 DeviceMajor:0 DeviceMinor:316 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/295d3d68-c2d1-497f-9a6b-7bd5017b2a41/volumes/kubernetes.io~projected/kube-api-access-swmmr DeviceMajor:0 DeviceMinor:60 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6c5cd5b02cd04f136ddd35d63270258c503b774db5dab8c813d36fd52facf463/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cfc15e221e67bda02a6c2e74fcf4471435147084bb0ab22024fcdb611dc521a1/userdata/shm DeviceMajor:0 DeviceMinor:330 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42fd8ec3-88cb-4e0b-9d58-1c654ff20582/volumes/kubernetes.io~projected/kube-api-access-vrvmm DeviceMajor:0 DeviceMinor:202 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/567008ed-4e09-47b1-b76b-050e148b71d7/volumes/kubernetes.io~projected/kube-api-access-v4mbd DeviceMajor:0 DeviceMinor:327 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/1dde0398-5d23-49ed-b073-ddf5a2984678/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:354 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582164480 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/bf06a906-365c-44b3-b3c2-75a74fc8fdf0/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:56 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814/userdata/shm DeviceMajor:0 DeviceMinor:47 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes/kubernetes.io~projected/kube-api-access-w2sbv DeviceMajor:0 DeviceMinor:220 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c6a49d28-717c-4fab-91be-c77a2e58b037/volumes/kubernetes.io~projected/kube-api-access-7xqks DeviceMajor:0 DeviceMinor:222 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/729125a8c1f724ad90522448c0cedd08e7c96f8d3480af183aff99252aee638b/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:210 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-265 DeviceMajor:0 DeviceMinor:265 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8a3164b1547d0a3f20c42b72af4e0e93abf95444eed052fc28aa43293af14f90/userdata/shm DeviceMajor:0 DeviceMinor:296 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-113 DeviceMajor:0 DeviceMinor:113 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cc378d17-492d-4255-bd20-998d518e8b04/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:208 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/304f876e-02df-45da-af55-62cae8468a50/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:288 Capacity:31985729536 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-302 DeviceMajor:0 DeviceMinor:302 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:123fa29e13b0235 MacAddress:46:34:7c:02:e3:2d Speed:10000 Mtu:8901} {Name:1d1018c05abc585 MacAddress:1a:86:d1:4d:07:e4 Speed:10000 Mtu:8901} {Name:302f89b9f3c38df MacAddress:42:09:9c:05:a1:54 Speed:10000 Mtu:8901} {Name:54a8069f0fe8219 MacAddress:1a:0f:e3:77:1e:d5 Speed:10000 Mtu:8901} {Name:5a2f0d1d4c858ec MacAddress:2e:54:18:d4:50:d2 Speed:10000 Mtu:8901} {Name:6344759d70c3f54 MacAddress:9a:bb:5e:22:9c:b7 Speed:10000 Mtu:8901} {Name:64628b709fe46d4 MacAddress:62:63:2a:c7:f9:82 Speed:10000 Mtu:8901} {Name:6d7588ae89226c0 MacAddress:5e:5a:27:4b:00:0c Speed:10000 Mtu:8901} {Name:8a3164b1547d0a3 MacAddress:ea:c4:d7:fa:fc:4e Speed:10000 Mtu:8901} {Name:9572f83e8b46a46 MacAddress:72:58:a6:15:cd:2e Speed:10000 Mtu:8901} {Name:a879dc931a0686c MacAddress:f6:ee:7f:16:b3:c9 Speed:10000 Mtu:8901} {Name:a9d4979cabc5462 MacAddress:22:02:f4:68:ae:08 Speed:10000 Mtu:8901} {Name:b34ee95f4a012a0 MacAddress:2e:92:8a:eb:81:e4 Speed:10000 Mtu:8901} {Name:bbb1573e4f4f87d MacAddress:8a:38:29:53:c2:1e Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:ff:d0:d0:3a:d7 Speed:0 Mtu:9001} {Name:br-int MacAddress:9a:75:0d:eb:c4:df Speed:0 Mtu:8901} {Name:cfc15e221e67bda MacAddress:12:9f:dc:9a:ee:ed Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:ff:d0:d0:3a:d7 Speed:0 Mtu:9001} {Name:fd47b453a0add5f MacAddress:0e:47:0b:8a:ad:29 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:b6:53:2c:d8:ad:11 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:f6:a8:61:a4:b2:25 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164328960 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.332906    5057 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.332951    5057 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333280    5057 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333306    5057 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-148.us-east-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333461    5057 topology_manager.go:143] "Creating topology manager with none policy"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333470    5057 container_manager_linux.go:311] "Creating device plugin manager"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333480    5057 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333513    5057 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333537    5057 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333727    5057 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333840    5057 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333878    5057 kubelet.go:493] "Attempting to sync node with API server"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333891    5057 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333902    5057 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333911    5057 kubelet.go:405] "Adding apiserver pod source"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.333920    5057 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.334805    5057 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.334815    5057 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.335392    5057 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338115    5057 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338128    5057 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338238    5057 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338486    5057 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338552    5057 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338662    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338679    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338689    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338697    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338706    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338715    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338724    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338734    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338744    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338753    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338772    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338787    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338820    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.338828    5057 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.343213    5057 server.go:1263] "Started kubelet"
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.343301    5057 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.343520    5057 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Nov 01 22:22:23.343668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.343587    5057 server_v1.go:49] "podresources" method="list" useActivePods=true
Nov 01 22:22:23.345413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.344014    5057 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Nov 01 22:22:23.343743 ip-10-0-141-148 systemd[1]: Started Kubernetes Kubelet.
Nov 01 22:22:23.352715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.352300    5057 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:22:23.352715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.352528    5057 server.go:317] "Adding debug handlers to kubelet server"
Nov 01 22:22:23.355039 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.354967    5057 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:22:23.356360 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.356344    5057 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Nov 01 22:22:23.356536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.356480    5057 volume_manager.go:309] "The desired_state_of_world populator starts"
Nov 01 22:22:23.356536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.356518    5057 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Nov 01 22:22:23.356640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.356539    5057 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Nov 01 22:22:23.356843 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.356524    5057 volume_manager.go:311] "Starting Kubelet Volume Manager"
Nov 01 22:22:23.357607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.357556    5057 factory.go:55] Registering systemd factory
Nov 01 22:22:23.357607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.357579    5057 factory.go:223] Registration of the systemd container factory successfully
Nov 01 22:22:23.357746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.357622    5057 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-31 22:16:42 +0000 UTC" deadline="2028-06-28 01:01:54.965224676 +0000 UTC"
Nov 01 22:22:23.357746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.357644    5057 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14498h39m31.607583358s"
Nov 01 22:22:23.358763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.358742    5057 factory.go:153] Registering CRI-O factory
Nov 01 22:22:23.358846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.358774    5057 factory.go:223] Registration of the crio container factory successfully
Nov 01 22:22:23.358911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.358900    5057 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Nov 01 22:22:23.358964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.358923    5057 factory.go:103] Registering Raw factory
Nov 01 22:22:23.358964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.358939    5057 manager.go:1199] Started watching for new ooms in manager
Nov 01 22:22:23.359371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.359355    5057 manager.go:319] Starting recovery of all containers
Nov 01 22:22:23.360324 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360300    5057 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:22:23.360550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360138    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1dde0398-5d23-49ed-b073-ddf5a2984678" volumeName="kubernetes.io/secret/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-key" seLinuxMountContext=""
Nov 01 22:22:23.360621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360572    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42fd8ec3-88cb-4e0b-9d58-1c654ff20582" volumeName="kubernetes.io/projected/42fd8ec3-88cb-4e0b-9d58-1c654ff20582-kube-api-access-vrvmm" seLinuxMountContext=""
Nov 01 22:22:23.360621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360598    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates" seLinuxMountContext=""
Nov 01 22:22:23.360722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360622    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca" seLinuxMountContext=""
Nov 01 22:22:23.360722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360639    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token" seLinuxMountContext=""
Nov 01 22:22:23.360722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360652    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted" seLinuxMountContext=""
Nov 01 22:22:23.360722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360671    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91a8683-96c3-4725-b90b-da6590c5076e" volumeName="kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-snapshots" seLinuxMountContext=""
Nov 01 22:22:23.360722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360684    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets" seLinuxMountContext=""
Nov 01 22:22:23.360722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360709    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" volumeName="kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360724    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298f62f9-b298-474c-adec-6b780cca9db8" volumeName="kubernetes.io/projected/298f62f9-b298-474c-adec-6b780cca9db8-kube-api-access-nwpgl" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360744    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e7fb00c-4f60-4b4c-9402-65cc6ec7e887" volumeName="kubernetes.io/configmap/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-serviceca" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360757    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac40f60c-ca7d-48cf-92fe-a8fea30a5d54" volumeName="kubernetes.io/projected/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-kube-api-access-nqhph" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360766    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c80d6c27-cb56-46c7-bed5-b96e23274d9c" volumeName="kubernetes.io/configmap/c80d6c27-cb56-46c7-bed5-b96e23274d9c-konnectivity-ca" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360777    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91a8683-96c3-4725-b90b-da6590c5076e" volumeName="kubernetes.io/projected/d91a8683-96c3-4725-b90b-da6590c5076e-kube-api-access-7fq9b" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360802    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91a8683-96c3-4725-b90b-da6590c5076e" volumeName="kubernetes.io/secret/d91a8683-96c3-4725-b90b-da6590c5076e-serving-cert" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360815    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360825    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="295d3d68-c2d1-497f-9a6b-7bd5017b2a41" volumeName="kubernetes.io/projected/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-kube-api-access-swmmr" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360839    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="567008ed-4e09-47b1-b76b-050e148b71d7" volumeName="kubernetes.io/projected/567008ed-4e09-47b1-b76b-050e148b71d7-kube-api-access-v4mbd" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360848    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="71d83910-eb9d-4178-b1c3-3fdf24524dea" volumeName="kubernetes.io/configmap/71d83910-eb9d-4178-b1c3-3fdf24524dea-telemetry-config" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360858    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81511c2e-f1cb-40ea-8d52-b795feb55690" volumeName="kubernetes.io/projected/81511c2e-f1cb-40ea-8d52-b795feb55690-kube-api-access-7lg2q" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360869    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf06a906-365c-44b3-b3c2-75a74fc8fdf0" volumeName="kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-tmp" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360878    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c23ca1da-3563-4710-8fa7-6ed89d1b3e7f" volumeName="kubernetes.io/projected/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-kube-api-access-96spj" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360895    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="762e65a1-8e7f-4d15-945c-c29c442ac478" volumeName="kubernetes.io/projected/762e65a1-8e7f-4d15-945c-c29c442ac478-kube-api-access-zq227" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360904    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac40f60c-ca7d-48cf-92fe-a8fea30a5d54" volumeName="kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-stats-auth" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360914    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360926    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c6a49d28-717c-4fab-91be-c77a2e58b037" volumeName="kubernetes.io/projected/c6a49d28-717c-4fab-91be-c77a2e58b037-kube-api-access-7xqks" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360934    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360946    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="295d3d68-c2d1-497f-9a6b-7bd5017b2a41" volumeName="kubernetes.io/empty-dir/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-tmp-dir" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360964    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fae5b32c-a10a-442a-8d19-e5ab9221ed37" volumeName="kubernetes.io/configmap/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-config" seLinuxMountContext=""
Nov 01 22:22:23.360973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360973    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.360989    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" volumeName="kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361001    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298f62f9-b298-474c-adec-6b780cca9db8" volumeName="kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-binary-copy" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361011    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b5dfc1c-8c00-45aa-b941-64f36a914db3" volumeName="kubernetes.io/projected/4b5dfc1c-8c00-45aa-b941-64f36a914db3-kube-api-access-2cj6q" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361023    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81511c2e-f1cb-40ea-8d52-b795feb55690" volumeName="kubernetes.io/empty-dir/81511c2e-f1cb-40ea-8d52-b795feb55690-tmp-dir" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361032    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9918792e-83ee-41de-8794-bf799e14788d" volumeName="kubernetes.io/configmap/9918792e-83ee-41de-8794-bf799e14788d-nginx-conf" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361040    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c80d6c27-cb56-46c7-bed5-b96e23274d9c" volumeName="kubernetes.io/secret/c80d6c27-cb56-46c7-bed5-b96e23274d9c-agent-certs" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361051    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="304f876e-02df-45da-af55-62cae8468a50" volumeName="kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361060    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b5dfc1c-8c00-45aa-b941-64f36a914db3" volumeName="kubernetes.io/configmap/4b5dfc1c-8c00-45aa-b941-64f36a914db3-config" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361071    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="81511c2e-f1cb-40ea-8d52-b795feb55690" volumeName="kubernetes.io/configmap/81511c2e-f1cb-40ea-8d52-b795feb55690-config-volume" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361079    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf06a906-365c-44b3-b3c2-75a74fc8fdf0" volumeName="kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-tuned" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361090    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf01e723-91f6-4c22-8c8a-78b1cdd730af" volumeName="kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-config" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361104    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fae5b32c-a10a-442a-8d19-e5ab9221ed37" volumeName="kubernetes.io/projected/fae5b32c-a10a-442a-8d19-e5ab9221ed37-kube-api-access-w7sth" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361112    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbc3a17c-1c1d-405d-b062-e2b42b5e53e2" volumeName="kubernetes.io/projected/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-kube-api-access-2jstc" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361124    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361132    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361140    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d" volumeName="kubernetes.io/projected/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d-kube-api-access-ksq2c" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361152    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298f62f9-b298-474c-adec-6b780cca9db8" volumeName="kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-sysctl-allowlist" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361160    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7d5c4c29-2458-449a-9348-d36bef863863" volumeName="kubernetes.io/projected/7d5c4c29-2458-449a-9348-d36bef863863-kube-api-access-b4wsn" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361172    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc7341bb-795d-430a-a015-36004ec6723a" volumeName="kubernetes.io/configmap/cc7341bb-795d-430a-a015-36004ec6723a-iptables-alerter-script" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361181    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" volumeName="kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361190    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="514791bf-52ab-46d1-bec2-44baa1b3f48d" volumeName="kubernetes.io/projected/514791bf-52ab-46d1-bec2-44baa1b3f48d-kube-api-access-kc4xk" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361202    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="812cb630-c9f1-4b19-b5e2-b57fda496558" volumeName="kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-config" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361211    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361223    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="298f62f9-b298-474c-adec-6b780cca9db8" volumeName="kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-whereabouts-flatfile-configmap" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361232    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b5dfc1c-8c00-45aa-b941-64f36a914db3" volumeName="kubernetes.io/secret/4b5dfc1c-8c00-45aa-b941-64f36a914db3-serving-cert" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361241    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ac40f60c-ca7d-48cf-92fe-a8fea30a5d54" volumeName="kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-default-certificate" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361265    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb04c807-ee72-4833-a98b-689f42b18e4c" volumeName="kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361275    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf01e723-91f6-4c22-8c8a-78b1cdd730af" volumeName="kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-env-overrides" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361287    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91a8683-96c3-4725-b90b-da6590c5076e" volumeName="kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-tmp" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361296    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2f6f200-9701-4977-9256-68945f6568f2" volumeName="kubernetes.io/secret/f2f6f200-9701-4977-9256-68945f6568f2-serving-cert" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361305    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4e7fb00c-4f60-4b4c-9402-65cc6ec7e887" volumeName="kubernetes.io/projected/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-kube-api-access-98blt" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361316    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="812cb630-c9f1-4b19-b5e2-b57fda496558" volumeName="kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-trusted-ca" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361325    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb04c807-ee72-4833-a98b-689f42b18e4c" volumeName="kubernetes.io/empty-dir/cb04c807-ee72-4833-a98b-689f42b18e4c-data-volume" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361337    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1dde0398-5d23-49ed-b073-ddf5a2984678" volumeName="kubernetes.io/projected/1dde0398-5d23-49ed-b073-ddf5a2984678-kube-api-access-ggmlj" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361346    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="812cb630-c9f1-4b19-b5e2-b57fda496558" volumeName="kubernetes.io/projected/812cb630-c9f1-4b19-b5e2-b57fda496558-kube-api-access-4jr28" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361354    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="aed985ae-387f-4126-a690-6b42829acc8a" volumeName="kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361365    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc7341bb-795d-430a-a015-36004ec6723a" volumeName="kubernetes.io/projected/cc7341bb-795d-430a-a015-36004ec6723a-kube-api-access-4njbt" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361375    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf01e723-91f6-4c22-8c8a-78b1cdd730af" volumeName="kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-script-lib" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361386    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91a8683-96c3-4725-b90b-da6590c5076e" volumeName="kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-service-ca-bundle" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361396    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2f6f200-9701-4977-9256-68945f6568f2" volumeName="kubernetes.io/configmap/f2f6f200-9701-4977-9256-68945f6568f2-config" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361431    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f2f6f200-9701-4977-9256-68945f6568f2" volumeName="kubernetes.io/projected/f2f6f200-9701-4977-9256-68945f6568f2-kube-api-access-sjhgp" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361444    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1dde0398-5d23-49ed-b073-ddf5a2984678" volumeName="kubernetes.io/configmap/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-cabundle" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361454    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="812cb630-c9f1-4b19-b5e2-b57fda496558" volumeName="kubernetes.io/secret/812cb630-c9f1-4b19-b5e2-b57fda496558-serving-cert" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361468    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b546a0cd-894e-46ab-b097-6684578a546c" volumeName="kubernetes.io/projected/b546a0cd-894e-46ab-b097-6684578a546c-kube-api-access-ldv8d" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361480    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bf06a906-365c-44b3-b3c2-75a74fc8fdf0" volumeName="kubernetes.io/projected/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-kube-api-access-6fhbf" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361505    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf01e723-91f6-4c22-8c8a-78b1cdd730af" volumeName="kubernetes.io/secret/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovn-node-metrics-cert" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361524    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbc3a17c-1c1d-405d-b062-e2b42b5e53e2" volumeName="kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-daemon-config" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361533    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="fbc3a17c-1c1d-405d-b062-e2b42b5e53e2" volumeName="kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cni-binary-copy" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361574    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0895864a-960a-4fef-9f7e-8d56abeabbcd" volumeName="kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361584    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bfbadae-7483-48c4-9fe4-a178c2e3c15b" volumeName="kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361594    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cb04c807-ee72-4833-a98b-689f42b18e4c" volumeName="kubernetes.io/projected/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-api-access-szlnl" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361606    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc378d17-492d-4255-bd20-998d518e8b04" volumeName="kubernetes.io/projected/cc378d17-492d-4255-bd20-998d518e8b04-kube-api-access-kwfhk" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361616    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc378d17-492d-4255-bd20-998d518e8b04" volumeName="kubernetes.io/secret/cc378d17-492d-4255-bd20-998d518e8b04-cert" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361627    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cf01e723-91f6-4c22-8c8a-78b1cdd730af" volumeName="kubernetes.io/projected/cf01e723-91f6-4c22-8c8a-78b1cdd730af-kube-api-access-ms7s9" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361637    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="71d83910-eb9d-4178-b1c3-3fdf24524dea" volumeName="kubernetes.io/projected/71d83910-eb9d-4178-b1c3-3fdf24524dea-kube-api-access-9hj5f" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361645    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="d91a8683-96c3-4725-b90b-da6590c5076e" volumeName="kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-trusted-ca-bundle" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361658    5057 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" volumeName="kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle" seLinuxMountContext=""
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361666    5057 reconstruct.go:97] "Volume reconstruction finished"
Nov 01 22:22:23.361855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.361671    5057 reconciler.go:29] "Reconciler: start to sync state"
Nov 01 22:22:23.365906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.365852    5057 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Nov 01 22:22:23.369537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.369485    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:22:23.370674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.370658    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:22:23.387556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.387520    5057 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Nov 01 22:22:23.388835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.388813    5057 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Nov 01 22:22:23.388932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.388842    5057 status_manager.go:249] "Starting to sync pod status with apiserver"
Nov 01 22:22:23.388932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.388864    5057 kubelet.go:2521] "Starting kubelet main sync loop"
Nov 01 22:22:23.388932 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:23.388904    5057 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Nov 01 22:22:23.392886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.392864    5057 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Nov 01 22:22:23.434730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.434690    5057 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="5fb59a8b682125d4f379770c45a36ca1b0323b6979476ff259f709c759ed1a49" exitCode=0
Nov 01 22:22:23.434730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.434726    5057 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="1a04369ef6e21e4d590fc3e7276b146f86c926d8d18d39477e3f0f1d06135753" exitCode=0
Nov 01 22:22:23.434730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.434736    5057 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="3ff7fc2baf94ee76eebfee53d7124cbe3ad4e7e59b6b99cdfd647062645cc948" exitCode=0
Nov 01 22:22:23.434967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.434747    5057 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="e77fbc384211cb0f09c2517228d61bf075a4ea5c3b073c5a1602bccf809ddd42" exitCode=0
Nov 01 22:22:23.434967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.434755    5057 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="fbd8982eb350310d12d2bea5f49a9c50f0546b528fe3bc9fdbfdda08ad86dcb4" exitCode=0
Nov 01 22:22:23.434967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.434762    5057 generic.go:356] "Generic (PLEG): container finished" podID="298f62f9-b298-474c-adec-6b780cca9db8" containerID="361d92bcb85119ed29fda17dfde4c905f652e3cd20b6c6ba75504c2d4f7f8923" exitCode=0
Nov 01 22:22:23.440041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.440011    5057 generic.go:356] "Generic (PLEG): container finished" podID="564eced2d21b536583a4d40aec7fcb5f" containerID="8a51ad3a7cbcc0b3eb39f20c6cb3f57f36ebb1b01dd555c568b26fd950780fac" exitCode=0
Nov 01 22:22:23.443267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.443248    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:22:23.443376 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.443276    5057 generic.go:356] "Generic (PLEG): container finished" podID="812cb630-c9f1-4b19-b5e2-b57fda496558" containerID="e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d" exitCode=255
Nov 01 22:22:23.461702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.461682    5057 manager.go:324] Recovery completed
Nov 01 22:22:23.485454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.485429    5057 cpu_manager.go:226] "Starting" policy="none"
Nov 01 22:22:23.485454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.485448    5057 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Nov 01 22:22:23.485640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.485468    5057 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Nov 01 22:22:23.485693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.485648    5057 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Nov 01 22:22:23.486638 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.485664    5057 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Nov 01 22:22:23.486731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486640    5057 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Nov 01 22:22:23.486731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486652    5057 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Nov 01 22:22:23.486731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486661    5057 policy_none.go:50] "Start"
Nov 01 22:22:23.486731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486673    5057 memory_manager.go:187] "Starting memorymanager" policy="None"
Nov 01 22:22:23.486731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486687    5057 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Nov 01 22:22:23.486961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486878    5057 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Nov 01 22:22:23.486961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486890    5057 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Nov 01 22:22:23.486961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.486897    5057 policy_none.go:44] "Start"
Nov 01 22:22:23.488977 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:23.488956    5057 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Nov 01 22:22:23.491349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.491322    5057 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Nov 01 22:22:23.494827 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.494814    5057 manager.go:341] "Starting Device Plugin manager"
Nov 01 22:22:23.494894 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:23.494839    5057 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Nov 01 22:22:23.494894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.494847    5057 server.go:87] "Starting device plugin registration server"
Nov 01 22:22:23.495168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495152    5057 eviction_manager.go:194] "Eviction manager: starting control loop"
Nov 01 22:22:23.495228 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495167    5057 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Nov 01 22:22:23.495276 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495240    5057 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Nov 01 22:22:23.495400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495345    5057 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Nov 01 22:22:23.495400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495386    5057 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Nov 01 22:22:23.495400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495394    5057 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Nov 01 22:22:23.495561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.495443    5057 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-01T22:22:23.495372195Z","UUID":"4855c2a8-71fa-48cb-b89c-9924746d3694","Handler":null,"Name":"","Endpoint":""}
Nov 01 22:22:23.496710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.496692    5057 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Nov 01 22:22:23.496710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.496717    5057 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Nov 01 22:22:23.496956 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:23.496937    5057 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Nov 01 22:22:23.596303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.596272    5057 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Nov 01 22:22:23.597720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.597698    5057 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientMemory"
Nov 01 22:22:23.597720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.597730    5057 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasNoDiskPressure"
Nov 01 22:22:23.597882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.597740    5057 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-141-148.us-east-2.compute.internal" event="NodeHasSufficientPID"
Nov 01 22:22:23.597882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.597780    5057 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.611224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.611166    5057 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.611309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.611240    5057 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.689067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689013    5057 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"]
Nov 01 22:22:23.689685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689638    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"68cec1540c43f1cf2055316a4f5734e4","Type":"ContainerStarted","Data":"34fd0b2425e3d40fed86bb57f044ae985cef36c2117e575cdf693ff8120c67cc"}
Nov 01 22:22:23.689685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689687    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"68cec1540c43f1cf2055316a4f5734e4","Type":"ContainerStarted","Data":"9d857cbf32f03316d5ea51b465e4a4030f27d3ccc2aac2b32acc1235f1586814"}
Nov 01 22:22:23.689816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689722    5057 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.689816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689765    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"564eced2d21b536583a4d40aec7fcb5f","Type":"ContainerStarted","Data":"1570586369e4050df406c854269e44b0967d1e288b8515873fd65714a507607b"}
Nov 01 22:22:23.689816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689779    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"564eced2d21b536583a4d40aec7fcb5f","Type":"ContainerDied","Data":"8a51ad3a7cbcc0b3eb39f20c6cb3f57f36ebb1b01dd555c568b26fd950780fac"}
Nov 01 22:22:23.689816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689791    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal" event={"ID":"564eced2d21b536583a4d40aec7fcb5f","Type":"ContainerStarted","Data":"c5036802c7f5e3359690f1be84b0bd2982c67f15cfeec86042088ba4a26ca032"}
Nov 01 22:22:23.689965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.689902    5057 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.709881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.709854    5057 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 22:22:23.709988 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:23.709911    5057 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.717429 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.717402    5057 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 22:22:23.717560 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:23.717450    5057 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.768042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.768008    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.768042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.768041    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.768209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.768059    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68cec1540c43f1cf2055316a4f5734e4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"68cec1540c43f1cf2055316a4f5734e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.868862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.868789    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.868862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.868823    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.868862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.868842    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68cec1540c43f1cf2055316a4f5734e4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"68cec1540c43f1cf2055316a4f5734e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.869064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.868886    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.869064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.868891    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/68cec1540c43f1cf2055316a4f5734e4-config\") pod \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"68cec1540c43f1cf2055316a4f5734e4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:23.869064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:23.868892    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/564eced2d21b536583a4d40aec7fcb5f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" (UID: \"564eced2d21b536583a4d40aec7fcb5f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:24.334289 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.334255    5057 apiserver.go:52] "Watching apiserver"
Nov 01 22:22:24.344625 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.344600    5057 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Nov 01 22:22:24.345444 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.345421    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-lkzkr","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5","openshift-ingress-canary/ingress-canary-j85jx","openshift-network-diagnostics/network-check-target-lg7nw","openshift-insights/insights-runtime-extractor-f54gq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal","openshift-multus/multus-additional-cni-plugins-zx4lx","openshift-image-registry/image-registry-586fc54c5f-9zzlg","kube-system/konnectivity-agent-b9n6q","openshift-network-console/networking-console-plugin-86c478bb88-nk6b8","openshift-image-registry/image-registry-dfcdddc4b-szq25","openshift-multus/multus-wr5zt","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x","openshift-dns/dns-default-fzbm2","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5","openshift-ovn-kubernetes/ovnkube-node-h5gq2","kube-system/kas-connection-checker-685895c7c9-r6xqj","openshift-insights/insights-operator-7c4986d449-fvxj8","openshift-monitoring/cluster-monitoring-operator-884576c57-l4759","kube-system/kas-connection-checker-685895c7c9-rxq5k","kube-system/global-pull-secret-syncer-9s2pz","openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2","openshift-network-operator/iptables-alerter-nwqgx","openshift-ingress/router-default-8444bb9d4c-m4lpd","openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74","openshift-service-ca/service-ca-5fffc97bdf-d47tp","openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd","openshift-multus/network-metrics-daemon-6gpsh","openshift-dns/node-resolver-9xfht","openshift-console-operator/console-operator-7cd56b8dbc-zz5p7","kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal","openshift-insights/periodic-gathering-75nhs-z7sqb","openshift-image-registry/node-ca-vjxrr","openshift-cluster-node-tuning-operator/tuned-6zmgx","openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"]
Nov 01 22:22:24.347759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.347741    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.348215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.348198    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.348539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.348523    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.349477 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.349447    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.349588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.349563    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.350259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.350239    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.350404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.350386    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.350587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.350571    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.350668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.350593    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.350908 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.350890    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Nov 01 22:22:24.351046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.351022    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.351132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.351113    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.351450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.351432    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Nov 01 22:22:24.351736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.351719    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Nov 01 22:22:24.351896 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.351880    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Nov 01 22:22:24.352045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.352028    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.352220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.352077    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Nov 01 22:22:24.352374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.352350    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Nov 01 22:22:24.352534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.352464    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Nov 01 22:22:24.352683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.352668    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.352962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.352205    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.353174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.353155    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Nov 01 22:22:24.353278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.353181    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Nov 01 22:22:24.353394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.353160    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.353656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.353629    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Nov 01 22:22:24.354511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.354473    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.354583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.354537    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Nov 01 22:22:24.354924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.354906    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.355317 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.355301    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Nov 01 22:22:24.355479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.355463    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Nov 01 22:22:24.356413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.356397    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.356476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.356416    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.356476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.356433    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Nov 01 22:22:24.356476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.356444    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Nov 01 22:22:24.358721 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.358699    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.358804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.358769    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Nov 01 22:22:24.358872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.358776    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.358925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.358893    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Nov 01 22:22:24.358925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.358909    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\""
Nov 01 22:22:24.359008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.358957    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.359077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.359060    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Nov 01 22:22:24.359178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.359081    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Nov 01 22:22:24.359178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.359103    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.359663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.359649    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:24.360239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.360222    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Nov 01 22:22:24.361388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.361375    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.362213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362193    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.362293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362214    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Nov 01 22:22:24.362293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362251    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.362390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362296    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Nov 01 22:22:24.362390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362331    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.362390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362357    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.362390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362387    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Nov 01 22:22:24.362580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.362442    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.363116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.363088    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Nov 01 22:22:24.363249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.363230    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Nov 01 22:22:24.363405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.363389    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.363522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.363485    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-79k59\""
Nov 01 22:22:24.363988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.363965    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-5scqq\""
Nov 01 22:22:24.363988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.363983    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-qjk9v\""
Nov 01 22:22:24.364212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364192    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Nov 01 22:22:24.364264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364211    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Nov 01 22:22:24.364264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364242    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-p26bd\""
Nov 01 22:22:24.364352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364206    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Nov 01 22:22:24.364407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364357    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Nov 01 22:22:24.364407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364394    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Nov 01 22:22:24.364546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364510    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-fpxpg\""
Nov 01 22:22:24.364546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364518    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Nov 01 22:22:24.364683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364662    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-4hpp8\""
Nov 01 22:22:24.364918 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364896    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.364979 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364933    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.365044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.364938    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zwl5x\""
Nov 01 22:22:24.365181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365167    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Nov 01 22:22:24.365230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365196    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.365456 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365437    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.365635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365617    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.365695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365643    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Nov 01 22:22:24.365802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365721    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Nov 01 22:22:24.365802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365741    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.365802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365790    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Nov 01 22:22:24.365936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365827    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Nov 01 22:22:24.365936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365867    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9pl6q\""
Nov 01 22:22:24.365936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365451    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Nov 01 22:22:24.366070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.365617    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Nov 01 22:22:24.366208 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366189    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5847z\""
Nov 01 22:22:24.366254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366225    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-4jwn7\""
Nov 01 22:22:24.366299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366271    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.366345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366226    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vvcmq\""
Nov 01 22:22:24.366345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366324    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dclh4\""
Nov 01 22:22:24.366432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366360    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.366432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366386    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j5zd8\""
Nov 01 22:22:24.366432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366403    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.366584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366455    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vv5lb\""
Nov 01 22:22:24.366584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366473    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.366584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366568    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Nov 01 22:22:24.366806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366789    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.366959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.366946    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.369956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.369936    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.370206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.370193    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.370472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.370459    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Nov 01 22:22:24.370540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.370505    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Nov 01 22:22:24.370671 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.370526    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4kglz\""
Nov 01 22:22:24.370937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.370924    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.371247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.371231    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.371389 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.371375    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ffzxb\""
Nov 01 22:22:24.372039 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372021    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kc4xk\" (UniqueName: \"kubernetes.io/projected/514791bf-52ab-46d1-bec2-44baa1b3f48d-kube-api-access-kc4xk\") pod \"kas-connection-checker-685895c7c9-r6xqj\" (UID: \"514791bf-52ab-46d1-bec2-44baa1b3f48d\") " pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:24.372075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372055    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91a8683-96c3-4725-b90b-da6590c5076e-serving-cert\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.372114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372080    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq227\" (UniqueName: \"kubernetes.io/projected/762e65a1-8e7f-4d15-945c-c29c442ac478-kube-api-access-zq227\") pod \"network-check-source-6f78d59f99-4gg4t\" (UID: \"762e65a1-8e7f-4d15-945c-c29c442ac478\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:24.372155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372114    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/812cb630-c9f1-4b19-b5e2-b57fda496558-serving-cert\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.372155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372144    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-system-cni-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.372226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372167    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.372226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372193    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjhgp\" (UniqueName: \"kubernetes.io/projected/f2f6f200-9701-4977-9256-68945f6568f2-kube-api-access-sjhgp\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:24.372306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372225    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.372339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372321    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dc544\""
Nov 01 22:22:24.372376 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372343    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.372422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372378    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-kubelet\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.372422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372396    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwpgl\" (UniqueName: \"kubernetes.io/projected/298f62f9-b298-474c-adec-6b780cca9db8-kube-api-access-nwpgl\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.372531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372413    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.372531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372517    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2f6f200-9701-4977-9256-68945f6568f2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:24.372640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372562    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b5dfc1c-8c00-45aa-b941-64f36a914db3-serving-cert\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:24.372640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372584    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Nov 01 22:22:24.372640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372582    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d91a8683-96c3-4725-b90b-da6590c5076e-serving-cert\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.372640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372623    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372647    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-k8s-cni-cncf-io\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372669    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-multus-certs\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372693    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372715    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/4b5dfc1c-8c00-45aa-b941-64f36a914db3-serving-cert\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372736    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrvmm\" (UniqueName: \"kubernetes.io/projected/42fd8ec3-88cb-4e0b-9d58-1c654ff20582-kube-api-access-vrvmm\") pod \"kas-connection-checker-685895c7c9-lkzkr\" (UID: \"42fd8ec3-88cb-4e0b-9d58-1c654ff20582\") " pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372772    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372789    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/812cb630-c9f1-4b19-b5e2-b57fda496558-serving-cert\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.372826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372808    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372834    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fq9b\" (UniqueName: \"kubernetes.io/projected/d91a8683-96c3-4725-b90b-da6590c5076e-kube-api-access-7fq9b\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372792    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372879    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4bt99\""
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372891    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f2f6f200-9701-4977-9256-68945f6568f2-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372893    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372920    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swmmr\" (UniqueName: \"kubernetes.io/projected/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-kube-api-access-swmmr\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372905    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.372978    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-daemon-config\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373057    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-trusted-ca\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373103    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-system-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373132    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-netns\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373204    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373226    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-conf-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373255    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-host\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373285    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cj6q\" (UniqueName: \"kubernetes.io/projected/4b5dfc1c-8c00-45aa-b941-64f36a914db3-kube-api-access-2cj6q\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373315    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-default-certificate\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373329    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373341    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/71d83910-eb9d-4178-b1c3-3fdf24524dea-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373372    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373375    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-daemon-config\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373396    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jstc\" (UniqueName: \"kubernetes.io/projected/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-kube-api-access-2jstc\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373431    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373432    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-trusted-ca\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373532    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-multus\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373562    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2f6f200-9701-4977-9256-68945f6568f2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373589    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-default-certificate\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373588    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-config\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373631    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-socket-dir-parent\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373640    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373658    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/71d83910-eb9d-4178-b1c3-3fdf24524dea-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373712    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-v4m7g\""
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373716    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Nov 01 22:22:24.373741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373753    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-service-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373727    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373772    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373805    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f2f6f200-9701-4977-9256-68945f6568f2-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373813    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/812cb630-c9f1-4b19-b5e2-b57fda496558-config\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373854    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-tmp-dir\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373878    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-serviceca\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373895    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhph\" (UniqueName: \"kubernetes.io/projected/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-kube-api-access-nqhph\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373915    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cnibin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373917    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-service-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373919    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-tmp-dir\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.373940    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksq2c\" (UniqueName: \"kubernetes.io/projected/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d-kube-api-access-ksq2c\") pod \"kas-connection-checker-685895c7c9-rxq5k\" (UID: \"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d\") " pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374046    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374078    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374115    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-hosts-file\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374143    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9918792e-83ee-41de-8794-bf799e14788d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374175    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374214    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xqks\" (UniqueName: \"kubernetes.io/projected/c6a49d28-717c-4fab-91be-c77a2e58b037-kube-api-access-7xqks\") pod \"volume-data-source-validator-58c9f5f46d-pnqc5\" (UID: \"c6a49d28-717c-4fab-91be-c77a2e58b037\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374258    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-snapshots\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374287    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7sth\" (UniqueName: \"kubernetes.io/projected/fae5b32c-a10a-442a-8d19-e5ab9221ed37-kube-api-access-w7sth\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374303    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-snapshots\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374314    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374325    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374326    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d91a8683-96c3-4725-b90b-da6590c5076e-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374360    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9918792e-83ee-41de-8794-bf799e14788d-nginx-conf\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374365    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374401    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-tmp\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374457    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-hostroot\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374536    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-stats-auth\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374562    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cni-binary-copy\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.374555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374471    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d91a8683-96c3-4725-b90b-da6590c5076e-tmp\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374583    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374609    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374634    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-bin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374658    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-cnibin\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374677    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-serviceca\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374683    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374716    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cni-binary-copy\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374737    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-stats-auth\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374737    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98blt\" (UniqueName: \"kubernetes.io/projected/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-kube-api-access-98blt\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374776    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hj5f\" (UniqueName: \"kubernetes.io/projected/71d83910-eb9d-4178-b1c3-3fdf24524dea-kube-api-access-9hj5f\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374888    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374901    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/298f62f9-b298-474c-adec-6b780cca9db8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374911    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.374949    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jr28\" (UniqueName: \"kubernetes.io/projected/812cb630-c9f1-4b19-b5e2-b57fda496558-kube-api-access-4jr28\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.375022    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.375045    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-etc-kubernetes\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.375083    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.375138    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b5dfc1c-8c00-45aa-b941-64f36a914db3-config\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.375230    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.375717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.375280    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/4b5dfc1c-8c00-45aa-b941-64f36a914db3-config\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:24.377904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.377885    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kc4xk\" (UniqueName: \"kubernetes.io/projected/514791bf-52ab-46d1-bec2-44baa1b3f48d-kube-api-access-kc4xk\") pod \"kas-connection-checker-685895c7c9-r6xqj\" (UID: \"514791bf-52ab-46d1-bec2-44baa1b3f48d\") " pod="kube-system/kas-connection-checker-685895c7c9-r6xqj"
Nov 01 22:22:24.378739 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.378713    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq227\" (UniqueName: \"kubernetes.io/projected/762e65a1-8e7f-4d15-945c-c29c442ac478-kube-api-access-zq227\") pod \"network-check-source-6f78d59f99-4gg4t\" (UID: \"762e65a1-8e7f-4d15-945c-c29c442ac478\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-4gg4t"
Nov 01 22:22:24.378829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.378745    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwpgl\" (UniqueName: \"kubernetes.io/projected/298f62f9-b298-474c-adec-6b780cca9db8-kube-api-access-nwpgl\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.378829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.378764    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrvmm\" (UniqueName: \"kubernetes.io/projected/42fd8ec3-88cb-4e0b-9d58-1c654ff20582-kube-api-access-vrvmm\") pod \"kas-connection-checker-685895c7c9-lkzkr\" (UID: \"42fd8ec3-88cb-4e0b-9d58-1c654ff20582\") " pod="kube-system/kas-connection-checker-685895c7c9-lkzkr"
Nov 01 22:22:24.378829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.378797    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjhgp\" (UniqueName: \"kubernetes.io/projected/f2f6f200-9701-4977-9256-68945f6568f2-kube-api-access-sjhgp\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-7stm5\" (UID: \"f2f6f200-9701-4977-9256-68945f6568f2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5"
Nov 01 22:22:24.378998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.378983    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fq9b\" (UniqueName: \"kubernetes.io/projected/d91a8683-96c3-4725-b90b-da6590c5076e-kube-api-access-7fq9b\") pod \"insights-operator-7c4986d449-fvxj8\" (UID: \"d91a8683-96c3-4725-b90b-da6590c5076e\") " pod="openshift-insights/insights-operator-7c4986d449-fvxj8"
Nov 01 22:22:24.380119 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.380104    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jstc\" (UniqueName: \"kubernetes.io/projected/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-kube-api-access-2jstc\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.380174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.380120    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cj6q\" (UniqueName: \"kubernetes.io/projected/4b5dfc1c-8c00-45aa-b941-64f36a914db3-kube-api-access-2cj6q\") pod \"service-ca-operator-544d749c8c-q2lj8\" (UID: \"4b5dfc1c-8c00-45aa-b941-64f36a914db3\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8"
Nov 01 22:22:24.380174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.380129    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swmmr\" (UniqueName: \"kubernetes.io/projected/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-kube-api-access-swmmr\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.381360 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.381336    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhph\" (UniqueName: \"kubernetes.io/projected/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-kube-api-access-nqhph\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.381652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.381623    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98blt\" (UniqueName: \"kubernetes.io/projected/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-kube-api-access-98blt\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.381834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.381810    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7sth\" (UniqueName: \"kubernetes.io/projected/fae5b32c-a10a-442a-8d19-e5ab9221ed37-kube-api-access-w7sth\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.381924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.381906    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksq2c\" (UniqueName: \"kubernetes.io/projected/1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d-kube-api-access-ksq2c\") pod \"kas-connection-checker-685895c7c9-rxq5k\" (UID: \"1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d\") " pod="kube-system/kas-connection-checker-685895c7c9-rxq5k"
Nov 01 22:22:24.381982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.381965    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xqks\" (UniqueName: \"kubernetes.io/projected/c6a49d28-717c-4fab-91be-c77a2e58b037-kube-api-access-7xqks\") pod \"volume-data-source-validator-58c9f5f46d-pnqc5\" (UID: \"c6a49d28-717c-4fab-91be-c77a2e58b037\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-pnqc5"
Nov 01 22:22:24.382050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.382032    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.382111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.382094    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jr28\" (UniqueName: \"kubernetes.io/projected/812cb630-c9f1-4b19-b5e2-b57fda496558-kube-api-access-4jr28\") pod \"console-operator-7cd56b8dbc-zz5p7\" (UID: \"812cb630-c9f1-4b19-b5e2-b57fda496558\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:24.382335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.382319    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hj5f\" (UniqueName: \"kubernetes.io/projected/71d83910-eb9d-4178-b1c3-3fdf24524dea-kube-api-access-9hj5f\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.382562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.382548    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.454357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.454333    5057 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:24.454691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.454666    5057 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:24.458380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.458359    5057 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Nov 01 22:22:24.465665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.465651    5057 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 22:22:24.465733 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.465693    5057 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:24.466471 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.466451    5057 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Nov 01 22:22:24.466595 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.466517    5057 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-148.us-east-2.compute.internal"
Nov 01 22:22:24.475474 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475454    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.475566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475482    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-systemd\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.475566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475519    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-netns\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.475566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475543    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-system-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.475667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475596    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovn-node-metrics-cert\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.475667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475603    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-system-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.475667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475639    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96spj\" (UniqueName: \"kubernetes.io/projected/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-kube-api-access-96spj\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:24.475766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475748    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.475804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475769    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovn-node-metrics-cert\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.475804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475796    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-conf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.475907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475830    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.475907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475879    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-cni-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.475907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475859    5057 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475910    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-multus\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475955    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475981    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.475983    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-multus\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476009    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-sys-fs\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476035    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-slash\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.476052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476041    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476058    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-key\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476097    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-cabundle\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476205    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cnibin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476239    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-cabundle\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476246    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-device-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476272    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-var-lib-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476282    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-cnibin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476297    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-kubernetes\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476326    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.476355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476336    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476374    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7341bb-795d-430a-a015-36004ec6723a-host-slash\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476373    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/1dde0398-5d23-49ed-b073-ddf5a2984678-signing-key\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476457    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-run\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476467    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476521    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-var-lib-kubelet\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476552    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476570    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476590    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c80d6c27-cb56-46c7-bed5-b96e23274d9c-konnectivity-ca\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476604    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-hostroot\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476623    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476646    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-bin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476656    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-hostroot\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476672    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476697    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/81511c2e-f1cb-40ea-8d52-b795feb55690-config-volume\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476702    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-cni-bin\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476729    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc378d17-492d-4255-bd20-998d518e8b04-cert\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476756    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwfhk\" (UniqueName: \"kubernetes.io/projected/cc378d17-492d-4255-bd20-998d518e8b04-kube-api-access-kwfhk\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:24.476774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476783    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476806    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-node-log\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476829    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-sys\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476854    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-etc-kubernetes\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476853    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c80d6c27-cb56-46c7-bed5-b96e23274d9c-konnectivity-ca\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476724    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476878    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/81511c2e-f1cb-40ea-8d52-b795feb55690-config-volume\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476878    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-ovn\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476939    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.476953    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cc378d17-492d-4255-bd20-998d518e8b04-cert\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477058    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477081    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-env-overrides\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477115    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/304f876e-02df-45da-af55-62cae8468a50-original-pull-secret\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477157    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-etc-kubernetes\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477211    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c80d6c27-cb56-46c7-bed5-b96e23274d9c-agent-certs\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477242    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477313    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-env-overrides\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477365    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477400    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c80d6c27-cb56-46c7-bed5-b96e23274d9c-agent-certs\") pod \"konnectivity-agent-b9n6q\" (UID: \"c80d6c27-cb56-46c7-bed5-b96e23274d9c\") " pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477427    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.477530 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.477452    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:24.977431255 +0000 UTC m=+1.719126874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477566    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ms7s9\" (UniqueName: \"kubernetes.io/projected/cf01e723-91f6-4c22-8c8a-78b1cdd730af-kube-api-access-ms7s9\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477630    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477763    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-k8s-cni-cncf-io\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477799    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-multus-certs\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477831    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477842    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-k8s-cni-cncf-io\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477887    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-multus-certs\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477928    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.477999    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-netns\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478019    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478036    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb04c807-ee72-4833-a98b-689f42b18e4c-crio-socket\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478052    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478073    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-conf-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478088    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-host\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478104    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb04c807-ee72-4833-a98b-689f42b18e4c-data-volume\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478121    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478127    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggmlj\" (UniqueName: \"kubernetes.io/projected/1dde0398-5d23-49ed-b073-ddf5a2984678-kube-api-access-ggmlj\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478153    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-kubelet-config\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478242    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cb04c807-ee72-4833-a98b-689f42b18e4c-data-volume\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478399    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478399    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-conf-dir\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478425    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478425    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.478468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478438    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-run-netns\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478486    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4mbd\" (UniqueName: \"kubernetes.io/projected/567008ed-4e09-47b1-b76b-050e148b71d7-kube-api-access-v4mbd\") pod \"migrator-66df5d496b-nbpf2\" (UID: \"567008ed-4e09-47b1-b76b-050e148b71d7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478521    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-szlnl\" (UniqueName: \"kubernetes.io/projected/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-api-access-szlnl\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478536    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e7fb00c-4f60-4b4c-9402-65cc6ec7e887-host\") pod \"node-ca-vjxrr\" (UID: \"4e7fb00c-4f60-4b4c-9402-65cc6ec7e887\") " pod="openshift-image-registry/node-ca-vjxrr"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478569    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-etc-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478593    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-tuned\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478627    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-systemd-units\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478645    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-socket-dir-parent\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478685    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-tuned\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478800    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-multus-socket-dir-parent\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478857    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478901    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478932    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.478987    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-hosts-file\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479025    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479082    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-socket-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479110    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-registration-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479137    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479200    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-log-socket\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479228    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-script-lib\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479254    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-modprobe-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479280    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-lib-modules\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479308    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-systemd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.479335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479338    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4wsn\" (UniqueName: \"kubernetes.io/projected/7d5c4c29-2458-449a-9348-d36bef863863-kube-api-access-b4wsn\") pod \"migrator-66df5d496b-h4d74\" (UID: \"7d5c4c29-2458-449a-9348-d36bef863863\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479390    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479488    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-cnibin\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479541    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-bin\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479567    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysconfig\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479570    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479599    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fhbf\" (UniqueName: \"kubernetes.io/projected/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-kube-api-access-6fhbf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479633    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/295d3d68-c2d1-497f-9a6b-7bd5017b2a41-hosts-file\") pod \"node-resolver-9xfht\" (UID: \"295d3d68-c2d1-497f-9a6b-7bd5017b2a41\") " pod="openshift-dns/node-resolver-9xfht"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479649    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479784    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81511c2e-f1cb-40ea-8d52-b795feb55690-tmp-dir\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479818    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldv8d\" (UniqueName: \"kubernetes.io/projected/b546a0cd-894e-46ab-b097-6684578a546c-kube-api-access-ldv8d\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.479849    5057 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.479866    5057 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479972    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-cnibin\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480002    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-script-lib\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479849    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-kubelet\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.479850    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-metrics-certs\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480054    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/81511c2e-f1cb-40ea-8d52-b795feb55690-tmp-dir\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.480083    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:24.980045626 +0000 UTC m=+1.721741252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:24.480312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480268    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-netd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480309    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lg2q\" (UniqueName: \"kubernetes.io/projected/81511c2e-f1cb-40ea-8d52-b795feb55690-kube-api-access-7lg2q\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480528    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4njbt\" (UniqueName: \"kubernetes.io/projected/cc7341bb-795d-430a-a015-36004ec6723a-kube-api-access-4njbt\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480547    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/71d83910-eb9d-4178-b1c3-3fdf24524dea-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-l4759\" (UID: \"71d83910-eb9d-4178-b1c3-3fdf24524dea\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480603    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480616    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480628    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/fae5b32c-a10a-442a-8d19-e5ab9221ed37-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-v872x\" (UID: \"fae5b32c-a10a-442a-8d19-e5ab9221ed37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480630    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480762    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-config\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480795    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-host\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480823    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-dbus\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480937    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-system-cni-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.480981    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/298f62f9-b298-474c-adec-6b780cca9db8-system-cni-dir\") pod \"multus-additional-cni-plugins-zx4lx\" (UID: \"298f62f9-b298-474c-adec-6b780cca9db8\") " pod="openshift-multus/multus-additional-cni-plugins-zx4lx"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481014    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7341bb-795d-430a-a015-36004ec6723a-iptables-alerter-script\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481020    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cf01e723-91f6-4c22-8c8a-78b1cdd730af-ovnkube-config\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481060    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481095    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481103    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9918792e-83ee-41de-8794-bf799e14788d-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-nk6b8\" (UID: \"9918792e-83ee-41de-8794-bf799e14788d\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481136    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-kubelet\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481160    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fbc3a17c-1c1d-405d-b062-e2b42b5e53e2-host-var-lib-kubelet\") pod \"multus-wr5zt\" (UID: \"fbc3a17c-1c1d-405d-b062-e2b42b5e53e2\") " pod="openshift-multus/multus-wr5zt"
Nov 01 22:22:24.481174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481187    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-tmp\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.482190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481265    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-tmp\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.482190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481342    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/cc7341bb-795d-430a-a015-36004ec6723a-iptables-alerter-script\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.482190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.481590    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96spj\" (UniqueName: \"kubernetes.io/projected/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-kube-api-access-96spj\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:24.483063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.483042    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.483137 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.483084    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwfhk\" (UniqueName: \"kubernetes.io/projected/cc378d17-492d-4255-bd20-998d518e8b04-kube-api-access-kwfhk\") pod \"ingress-canary-j85jx\" (UID: \"cc378d17-492d-4255-bd20-998d518e8b04\") " pod="openshift-ingress-canary/ingress-canary-j85jx"
Nov 01 22:22:24.485664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.485646    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") pod \"periodic-gathering-75nhs-z7sqb\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") " pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:22:24.488996 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.488977    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.489235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.489219    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ms7s9\" (UniqueName: \"kubernetes.io/projected/cf01e723-91f6-4c22-8c8a-78b1cdd730af-kube-api-access-ms7s9\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.491330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.490695    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4wsn\" (UniqueName: \"kubernetes.io/projected/7d5c4c29-2458-449a-9348-d36bef863863-kube-api-access-b4wsn\") pod \"migrator-66df5d496b-h4d74\" (UID: \"7d5c4c29-2458-449a-9348-d36bef863863\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-h4d74"
Nov 01 22:22:24.491330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491112    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lg2q\" (UniqueName: \"kubernetes.io/projected/81511c2e-f1cb-40ea-8d52-b795feb55690-kube-api-access-7lg2q\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.491330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491204    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4njbt\" (UniqueName: \"kubernetes.io/projected/cc7341bb-795d-430a-a015-36004ec6723a-kube-api-access-4njbt\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.491330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491259    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-szlnl\" (UniqueName: \"kubernetes.io/projected/cb04c807-ee72-4833-a98b-689f42b18e4c-kube-api-access-szlnl\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.491523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491345    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4mbd\" (UniqueName: \"kubernetes.io/projected/567008ed-4e09-47b1-b76b-050e148b71d7-kube-api-access-v4mbd\") pod \"migrator-66df5d496b-nbpf2\" (UID: \"567008ed-4e09-47b1-b76b-050e148b71d7\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-nbpf2"
Nov 01 22:22:24.491523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491391    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggmlj\" (UniqueName: \"kubernetes.io/projected/1dde0398-5d23-49ed-b073-ddf5a2984678-kube-api-access-ggmlj\") pod \"service-ca-5fffc97bdf-d47tp\" (UID: \"1dde0398-5d23-49ed-b073-ddf5a2984678\") " pod="openshift-service-ca/service-ca-5fffc97bdf-d47tp"
Nov 01 22:22:24.491523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491508    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqjkn\" (UniqueName: \"kubernetes.io/projected/2bfbadae-7483-48c4-9fe4-a178c2e3c15b-kube-api-access-lqjkn\") pod \"network-check-target-lg7nw\" (UID: \"2bfbadae-7483-48c4-9fe4-a178c2e3c15b\") " pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:24.491793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491774    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldv8d\" (UniqueName: \"kubernetes.io/projected/b546a0cd-894e-46ab-b097-6684578a546c-kube-api-access-ldv8d\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.491839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.491777    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fhbf\" (UniqueName: \"kubernetes.io/projected/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-kube-api-access-6fhbf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582070    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-bin\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582096    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysconfig\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582140    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-bin\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582145    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysconfig\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582178    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582195    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-kubelet\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582209    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-netd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582226    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.582228    5057 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Nov 01 22:22:24.582255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582240    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582258    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-host\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.582270    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls podName:81511c2e-f1cb-40ea-8d52-b795feb55690 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:25.082256 +0000 UTC m=+1.823951610 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls") pod "dns-default-fzbm2" (UID: "81511c2e-f1cb-40ea-8d52-b795feb55690") : secret "dns-default-metrics-tls" not found
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582298    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-dbus\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582307    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-kubelet\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582326    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-cni-netd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.582332    5057 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.582373    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:22:25.082360367 +0000 UTC m=+1.824055986 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : secret "metrics-daemon-secret" not found
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582300    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-host\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582330    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.582387    5057 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582385    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-dbus\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582407    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582441    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-systemd\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582453    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-etc-selinux\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582461    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-netns\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.582476    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:25.08245964 +0000 UTC m=+1.824155257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582524    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-netns\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582526    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-systemd\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582538    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582551    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582564    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-conf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582573    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582594    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-sys-fs\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582616    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-slash\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582639    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-device-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.582630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582644    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-sys-fs\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582659    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-var-lib-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582680    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-kubernetes\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582686    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-slash\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582700    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-sysctl-conf\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582706    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7341bb-795d-430a-a015-36004ec6723a-host-slash\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582729    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-run\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582733    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-device-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582747    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-var-lib-kubelet\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582755    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-kubernetes\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582778    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cc7341bb-795d-430a-a015-36004ec6723a-host-slash\") pod \"iptables-alerter-nwqgx\" (UID: \"cc7341bb-795d-430a-a015-36004ec6723a\") " pod="openshift-network-operator/iptables-alerter-nwqgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582799    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-run\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582788    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582807    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-var-lib-kubelet\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582840    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-node-log\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582843    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582840    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-var-lib-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582862    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-sys\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582877    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-node-log\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582887    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-ovn\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582937    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb04c807-ee72-4833-a98b-689f42b18e4c-crio-socket\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582947    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-sys\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582967    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-kubelet-config\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.582983    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583000    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-etc-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583011    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cb04c807-ee72-4833-a98b-689f42b18e4c-crio-socket\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583021    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/304f876e-02df-45da-af55-62cae8468a50-kubelet-config\") pod \"global-pull-secret-syncer-9s2pz\" (UID: \"304f876e-02df-45da-af55-62cae8468a50\") " pod="kube-system/global-pull-secret-syncer-9s2pz"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583025    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-ovn\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583022    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-systemd-units\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583059    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-systemd-units\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583078    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.583088    5057 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583100    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-etc-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.583101    5057 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583119    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-openvswitch\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583128    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583103    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.583161    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:25.08315085 +0000 UTC m=+1.824846451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583204    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-socket-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583221    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-registration-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583242    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-log-socket\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583256    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-modprobe-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583269    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-lib-modules\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583296    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-registration-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583311    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-log-socket\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583324    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b546a0cd-894e-46ab-b097-6684578a546c-socket-dir\") pod \"aws-ebs-csi-driver-node-5wpvd\" (UID: \"b546a0cd-894e-46ab-b097-6684578a546c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5wpvd"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583353    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-systemd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583372    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-lib-modules\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583382    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bf06a906-365c-44b3-b3c2-75a74fc8fdf0-etc-modprobe-d\") pod \"tuned-6zmgx\" (UID: \"bf06a906-365c-44b3-b3c2-75a74fc8fdf0\") " pod="openshift-cluster-node-tuning-operator/tuned-6zmgx"
Nov 01 22:22:24.583841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.583432    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cf01e723-91f6-4c22-8c8a-78b1cdd730af-run-systemd\") pod \"ovnkube-node-h5gq2\" (UID: \"cf01e723-91f6-4c22-8c8a-78b1cdd730af\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:24.595283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.595224    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:24.648928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.648907    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"
Nov 01 22:22:24.651057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.651042    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"
Nov 01 22:22:24.654212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.654200    5057 scope.go:122] "RemoveContainer" containerID="e309b10ddd0d4d895e76e5a4a88f2535a07f5f918a438b0f2120ad70f97cbf8d"
Nov 01 22:22:24.655194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.655181    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"
Nov 01 22:22:24.691858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.691833    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-b9n6q"
Nov 01 22:22:24.951577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.951361    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-l4759"]
Nov 01 22:22:24.954173 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:24.954134    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71d83910_eb9d_4178_b1c3_3fdf24524dea.slice/crio-cc121b344bd9ee7f45cbee3553197dc7a746abe47e14dc35cfbf5e496a716dc2 WatchSource:0}: Error finding container cc121b344bd9ee7f45cbee3553197dc7a746abe47e14dc35cfbf5e496a716dc2: Status 404 returned error can't find the container with id cc121b344bd9ee7f45cbee3553197dc7a746abe47e14dc35cfbf5e496a716dc2
Nov 01 22:22:24.956635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.956617    5057 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 22:22:24.986737 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.986710    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:24.986919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.986898    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:24.987074 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.987058    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:25.987040368 +0000 UTC m=+2.728735983 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:24.987137 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.987085    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:24.987207 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.987194    5057 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:24.987257 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.987210    5057 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:24.987257 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:24.987245    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:25.987233801 +0000 UTC m=+2.728929399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:24.991428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:24.991405    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x"]
Nov 01 22:22:25.022832 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.022796    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-nk6b8"]
Nov 01 22:22:25.026536 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:25.026513    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9918792e_83ee_41de_8794_bf799e14788d.slice/crio-17280e5d4faec93d7a0de1b72154121d9f2135fb1581d952da424ae71fdb529a WatchSource:0}: Error finding container 17280e5d4faec93d7a0de1b72154121d9f2135fb1581d952da424ae71fdb529a: Status 404 returned error can't find the container with id 17280e5d4faec93d7a0de1b72154121d9f2135fb1581d952da424ae71fdb529a
Nov 01 22:22:25.087965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.087945    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:25.088079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.087997    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:25.088079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.088022    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:25.088079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.088055    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088097    5057 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088116    5057 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088155    5057 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088163    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:26.088147386 +0000 UTC m=+2.829842987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088175    5057 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088202    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:26.088185569 +0000 UTC m=+2.829881185 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:25.088240 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.088221    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:22:26.088209433 +0000 UTC m=+2.829905034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : secret "metrics-daemon-secret" not found
Nov 01 22:22:25.090220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.090200    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/81511c2e-f1cb-40ea-8d52-b795feb55690-metrics-tls\") pod \"dns-default-fzbm2\" (UID: \"81511c2e-f1cb-40ea-8d52-b795feb55690\") " pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:25.091814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.091779    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.196846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.196812    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.228225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.228165    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjxrr_4e7fb00c-4f60-4b4c-9402-65cc6ec7e887/node-ca/0.log"
Nov 01 22:22:25.267838 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.267803    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:25.331306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.331263    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.444831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.444794    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.458774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.458747    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759" event={"ID":"71d83910-eb9d-4178-b1c3-3fdf24524dea","Type":"ContainerStarted","Data":"cc121b344bd9ee7f45cbee3553197dc7a746abe47e14dc35cfbf5e496a716dc2"}
Nov 01 22:22:25.460958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.460938    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:22:25.461088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.461013    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7" event={"ID":"812cb630-c9f1-4b19-b5e2-b57fda496558","Type":"ContainerStarted","Data":"1bf7d52287464cd8a2bda8cc4a1bd72be1164cde8c1fdb0182b8098c980bed22"}
Nov 01 22:22:25.462292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.462267    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x" event={"ID":"fae5b32c-a10a-442a-8d19-e5ab9221ed37","Type":"ContainerStarted","Data":"ec7e3b882c011068c4b0053c10b6ab59e014d48e9ea01f3b0498d6651de388d0"}
Nov 01 22:22:25.463489 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.463462    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8" event={"ID":"9918792e-83ee-41de-8794-bf799e14788d","Type":"ContainerStarted","Data":"17280e5d4faec93d7a0de1b72154121d9f2135fb1581d952da424ae71fdb529a"}
Nov 01 22:22:25.538931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.538880    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-fzbm2"]
Nov 01 22:22:25.541975 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:25.541950    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81511c2e_f1cb_40ea_8d52_b795feb55690.slice/crio-64d66b434a23bf27bf8b22d6f3afd14d9aac8073ee7d8dce74ad44951fff7104 WatchSource:0}: Error finding container 64d66b434a23bf27bf8b22d6f3afd14d9aac8073ee7d8dce74ad44951fff7104: Status 404 returned error can't find the container with id 64d66b434a23bf27bf8b22d6f3afd14d9aac8073ee7d8dce74ad44951fff7104
Nov 01 22:22:25.557754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.557703    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.660063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.660033    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.788154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.788114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.829169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.829036    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j85jx_cc378d17-492d-4255-bd20-998d518e8b04/serve-healthcheck-canary/0.log"
Nov 01 22:22:25.902335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.902307    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:25.910108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.910071    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:25.995845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.995804    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:25.996026 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.995955    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle podName:ac40f60c-ca7d-48cf-92fe-a8fea30a5d54 nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.99593536 +0000 UTC m=+4.737630960 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle") pod "router-default-8444bb9d4c-m4lpd" (UID: "ac40f60c-ca7d-48cf-92fe-a8fea30a5d54") : configmap references non-existent config key: service-ca.crt
Nov 01 22:22:25.996260 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:25.996203    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:25.996419 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.996404    5057 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:25.996518 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.996422    5057 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-586fc54c5f-9zzlg: secret "image-registry-tls" not found
Nov 01 22:22:25.996518 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:25.996468    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls podName:aed985ae-387f-4126-a690-6b42829acc8a nodeName:}" failed. No retries permitted until 2026-11-01 22:22:27.996451399 +0000 UTC m=+4.738147015 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls") pod "image-registry-586fc54c5f-9zzlg" (UID: "aed985ae-387f-4126-a690-6b42829acc8a") : secret "image-registry-tls" not found
Nov 01 22:22:26.028903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.028873    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-h4d74_7d5c4c29-2458-449a-9348-d36bef863863/migrator/0.log"
Nov 01 22:22:26.063782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.063747    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:26.097636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.097568    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:26.097773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.097676    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:26.097773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.097713    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.097987    5057 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.098008    5057 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-dfcdddc4b-szq25: secret "image-registry-tls" not found
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.098010    5057 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.098074    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs podName:c23ca1da-3563-4710-8fa7-6ed89d1b3e7f nodeName:}" failed. No retries permitted until 2026-11-01 22:22:28.098055633 +0000 UTC m=+4.839751250 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs") pod "network-metrics-daemon-6gpsh" (UID: "c23ca1da-3563-4710-8fa7-6ed89d1b3e7f") : secret "metrics-daemon-secret" not found
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.098083    5057 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.098090    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls podName:0895864a-960a-4fef-9f7e-8d56abeabbcd nodeName:}" failed. No retries permitted until 2026-11-01 22:22:28.09808382 +0000 UTC m=+4.839779416 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls") pod "image-registry-dfcdddc4b-szq25" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd") : secret "image-registry-tls" not found
Nov 01 22:22:26.098359 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:26.098146    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:28.098129022 +0000 UTC m=+4.839824620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:26.229695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.229670    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-nbpf2_567008ed-4e09-47b1-b76b-050e148b71d7/migrator/0.log"
Nov 01 22:22:26.288726 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.288689    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:26.312663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.312641    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:26.399454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.399386    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:26.432169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.432130    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-7stm5_f2f6f200-9701-4977-9256-68945f6568f2/kube-storage-version-migrator-operator/0.log"
Nov 01 22:22:26.470996 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.470956    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-nk6b8" event={"ID":"9918792e-83ee-41de-8794-bf799e14788d","Type":"ContainerStarted","Data":"0fbd7af4bc7e85d785b64aacd544f6e3aed9464e5955c421c00833820aef4e86"}
Nov 01 22:22:26.472797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.472724    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fzbm2" event={"ID":"81511c2e-f1cb-40ea-8d52-b795feb55690","Type":"ContainerStarted","Data":"64d66b434a23bf27bf8b22d6f3afd14d9aac8073ee7d8dce74ad44951fff7104"}
Nov 01 22:22:26.472797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.472789    5057 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 22:22:26.584084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.584044    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:26.828548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.828512    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:26.931868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:26.931838    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:27.034188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.034099    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:27.190184 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.190058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:27.306996 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.306962    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:27.423294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.423232    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:27.480799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.480766    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-l4759" event={"ID":"71d83910-eb9d-4178-b1c3-3fdf24524dea","Type":"ContainerStarted","Data":"a8d42ffa76ec874639b3380a4efd2b518607d1cebcc554fdfc013bf46526de95"}
Nov 01 22:22:27.597940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.597896    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:27.798267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:27.798086    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:28.017978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.017948    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:28.018159 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.018026    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:28.018632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.018606    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac40f60c-ca7d-48cf-92fe-a8fea30a5d54-service-ca-bundle\") pod \"router-default-8444bb9d4c-m4lpd\" (UID: \"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54\") " pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:28.020679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.020650    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"image-registry-586fc54c5f-9zzlg\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") " pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:28.119415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.119382    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:28.119951 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.119428    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:28.119951 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.119530    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:28.119951 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:28.119638    5057 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:28.119951 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:28.119710    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls podName:cb04c807-ee72-4833-a98b-689f42b18e4c nodeName:}" failed. No retries permitted until 2026-11-01 22:22:32.119691368 +0000 UTC m=+8.861386981 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-f54gq" (UID: "cb04c807-ee72-4833-a98b-689f42b18e4c") : secret "insights-runtime-extractor-tls" not found
Nov 01 22:22:28.121960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.121938    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c23ca1da-3563-4710-8fa7-6ed89d1b3e7f-metrics-certs\") pod \"network-metrics-daemon-6gpsh\" (UID: \"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f\") " pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:28.122284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.122258    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"image-registry-dfcdddc4b-szq25\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") " pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:28.248574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.248548    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:28.248729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.248583    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:28.260681 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.260647    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-6gpsh"
Nov 01 22:22:28.262739 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.262676    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:28.401330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.401298    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:28.485127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.485098    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fzbm2" event={"ID":"81511c2e-f1cb-40ea-8d52-b795feb55690","Type":"ContainerStarted","Data":"d05d716f209fecf0ff2f55c15a4d65facdbfcbbc2b9d82cc00dbfe8263a3df8b"}
Nov 01 22:22:28.485529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.485164    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-fzbm2" event={"ID":"81511c2e-f1cb-40ea-8d52-b795feb55690","Type":"ContainerStarted","Data":"56f3b2979e5e704aa614148c29aa6ce21ab54b4efd2abc6d636887572f38c1fa"}
Nov 01 22:22:28.485529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.485230    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:28.486998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.486977    5057 generic.go:356] "Generic (PLEG): container finished" podID="d91a8683-96c3-4725-b90b-da6590c5076e" containerID="4584d47bd57acc759d2c58db24b08b4fbb0e8061e6c64c16ba773dd0460a445b" exitCode=0
Nov 01 22:22:28.487096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.487054    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" event={"ID":"d91a8683-96c3-4725-b90b-da6590c5076e","Type":"ContainerDied","Data":"4584d47bd57acc759d2c58db24b08b4fbb0e8061e6c64c16ba773dd0460a445b"}
Nov 01 22:22:28.487385 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.487369    5057 scope.go:122] "RemoveContainer" containerID="4584d47bd57acc759d2c58db24b08b4fbb0e8061e6c64c16ba773dd0460a445b"
Nov 01 22:22:28.489087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.489053    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x" event={"ID":"fae5b32c-a10a-442a-8d19-e5ab9221ed37","Type":"ContainerStarted","Data":"c680c1a4e9daf0bd44b45b302e2a9757dbb93a3d3d83b5452a9ed023858928a4"}
Nov 01 22:22:28.490752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.490728    5057 generic.go:356] "Generic (PLEG): container finished" podID="f2f6f200-9701-4977-9256-68945f6568f2" containerID="9432b4a54378ce03e9c28e06e14985cd497dc49e5910706d8bb3f47cb50a538d" exitCode=0
Nov 01 22:22:28.490849 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.490754    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5" event={"ID":"f2f6f200-9701-4977-9256-68945f6568f2","Type":"ContainerDied","Data":"9432b4a54378ce03e9c28e06e14985cd497dc49e5910706d8bb3f47cb50a538d"}
Nov 01 22:22:28.491219 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.491205    5057 scope.go:122] "RemoveContainer" containerID="9432b4a54378ce03e9c28e06e14985cd497dc49e5910706d8bb3f47cb50a538d"
Nov 01 22:22:28.566536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.566482    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-586fc54c5f-9zzlg"]
Nov 01 22:22:28.579084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.575913    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-8444bb9d4c-m4lpd"]
Nov 01 22:22:28.599716 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:28.598144    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac40f60c_ca7d_48cf_92fe_a8fea30a5d54.slice/crio-8025d19eeae0c728cee3baa14c13b8620f0bb9c16da59eed580b1eea69d81999 WatchSource:0}: Error finding container 8025d19eeae0c728cee3baa14c13b8620f0bb9c16da59eed580b1eea69d81999: Status 404 returned error can't find the container with id 8025d19eeae0c728cee3baa14c13b8620f0bb9c16da59eed580b1eea69d81999
Nov 01 22:22:28.599940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.599842    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-6gpsh"]
Nov 01 22:22:28.600084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.600034    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:28.600789 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.600768    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-dfcdddc4b-szq25"]
Nov 01 22:22:28.615944 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:28.615523    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0895864a_960a_4fef_9f7e_8d56abeabbcd.slice/crio-ca6f4fc7b0463d0012c8a52213b6e57093eb4dee11654bccc09577a12f862193 WatchSource:0}: Error finding container ca6f4fc7b0463d0012c8a52213b6e57093eb4dee11654bccc09577a12f862193: Status 404 returned error can't find the container with id ca6f4fc7b0463d0012c8a52213b6e57093eb4dee11654bccc09577a12f862193
Nov 01 22:22:28.985601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:28.985543    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:29.101628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.101587    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:29.212253 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.212218    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:29.270361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.270288    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:29.270525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.270420    5057 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 22:22:29.275608 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.275583    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-zz5p7"
Nov 01 22:22:29.401949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.401856    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:29.498176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.498144    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6gpsh" event={"ID":"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f","Type":"ContainerStarted","Data":"844d769653731c2fc3f2cf1542a2453f29874f150b1f6b57d0bdf7544e206b58"}
Nov 01 22:22:29.500069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.500040    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd" event={"ID":"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54","Type":"ContainerStarted","Data":"cf95ae6e7ec5ec3695e36f5bcbd71cbc06048a49e24049ea523a3938209bdbf1"}
Nov 01 22:22:29.500225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.500089    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd" event={"ID":"ac40f60c-ca7d-48cf-92fe-a8fea30a5d54","Type":"ContainerStarted","Data":"8025d19eeae0c728cee3baa14c13b8620f0bb9c16da59eed580b1eea69d81999"}
Nov 01 22:22:29.502059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.502039    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" event={"ID":"d91a8683-96c3-4725-b90b-da6590c5076e","Type":"ContainerStarted","Data":"826c0cd4bc000729604d8e7efae9b298400a59b85a23aa756217839aea8bfe21"}
Nov 01 22:22:29.504370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.504125    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-v872x" event={"ID":"fae5b32c-a10a-442a-8d19-e5ab9221ed37","Type":"ContainerStarted","Data":"67ac6e24b781611ca5da4b4910124175a55e8b2c0b804adc912ec94bc55609e6"}
Nov 01 22:22:29.506121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.506099    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg" event={"ID":"aed985ae-387f-4126-a690-6b42829acc8a","Type":"ContainerStarted","Data":"cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8"}
Nov 01 22:22:29.506199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.506137    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg" event={"ID":"aed985ae-387f-4126-a690-6b42829acc8a","Type":"ContainerStarted","Data":"8bd5b472937334253757e8c4eec061fef95447fca115820396260cba906a9289"}
Nov 01 22:22:29.508086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.508059    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-7stm5" event={"ID":"f2f6f200-9701-4977-9256-68945f6568f2","Type":"ContainerStarted","Data":"e8c8b6dcec2e1e5b0bc3324daf7380b81e976da68dcbfadd73f1adcfbd5893ca"}
Nov 01 22:22:29.509819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.509793    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25" event={"ID":"0895864a-960a-4fef-9f7e-8d56abeabbcd","Type":"ContainerStarted","Data":"1bcae5c2738020ee18db7de3a22eb7a5bcec050cad20cd9760c7ec83ea7b01d9"}
Nov 01 22:22:29.509931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.509840    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25" event={"ID":"0895864a-960a-4fef-9f7e-8d56abeabbcd","Type":"ContainerStarted","Data":"ca6f4fc7b0463d0012c8a52213b6e57093eb4dee11654bccc09577a12f862193"}
Nov 01 22:22:29.787705 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:29.787675    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:30.222050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.222018    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:30.248949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.248915    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:30.251629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.251606    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:30.517742 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.517657    5057 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 22:22:30.517742 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.517692    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6gpsh" event={"ID":"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f","Type":"ContainerStarted","Data":"e462ee1349e1dc7fc853c1f95e83a0bdc4a7013b3013b355e45937f1bfad6c3d"}
Nov 01 22:22:30.517742 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.517712    5057 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Nov 01 22:22:30.518268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.517745    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-6gpsh" event={"ID":"c23ca1da-3563-4710-8fa7-6ed89d1b3e7f","Type":"ContainerStarted","Data":"19d465a64c782481fa1defe4ad8b48bb88f00981dc6d26f469d1ac041f7c7146"}
Nov 01 22:22:30.518325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.518289    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:30.519212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.519193    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-8444bb9d4c-m4lpd"
Nov 01 22:22:30.588188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.588154    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:30.953971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.953944    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:30.988586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:30.988537    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:31.228235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:31.228160    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:31.381859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:31.381822    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:31.791008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:31.790976    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:32.164352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.164271    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:32.166681 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.166664    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cb04c807-ee72-4833-a98b-689f42b18e4c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-f54gq\" (UID: \"cb04c807-ee72-4833-a98b-689f42b18e4c\") " pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:32.170591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.170577    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-f54gq"
Nov 01 22:22:32.190427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.190392    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:32.394221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.394185    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:32.442407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.442385    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-f54gq"]
Nov 01 22:22:32.445046 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:32.445017    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb04c807_ee72_4833_a98b_689f42b18e4c.slice/crio-1fbf678ca99969506772da4a964a8c46425b30ec524fef39aca9504e1dcd4854 WatchSource:0}: Error finding container 1fbf678ca99969506772da4a964a8c46425b30ec524fef39aca9504e1dcd4854: Status 404 returned error can't find the container with id 1fbf678ca99969506772da4a964a8c46425b30ec524fef39aca9504e1dcd4854
Nov 01 22:22:32.532215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.532182    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f54gq" event={"ID":"cb04c807-ee72-4833-a98b-689f42b18e4c","Type":"ContainerStarted","Data":"4da19096a16c06e73a14af573a75c4ce1ef91493f12a151a3e653ab73e70e85c"}
Nov 01 22:22:32.532358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:32.532260    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f54gq" event={"ID":"cb04c807-ee72-4833-a98b-689f42b18e4c","Type":"ContainerStarted","Data":"1fbf678ca99969506772da4a964a8c46425b30ec524fef39aca9504e1dcd4854"}
Nov 01 22:22:33.537595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:33.537565    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f54gq" event={"ID":"cb04c807-ee72-4833-a98b-689f42b18e4c","Type":"ContainerStarted","Data":"c19d55784a85e984acc4d02101e5be0a86a9f5fbf3842832628557c87bd56695"}
Nov 01 22:22:33.754592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:33.754558    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:34.884704 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:34.884669    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:34.992575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:34.992533    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:35.551216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:35.551178    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-f54gq" event={"ID":"cb04c807-ee72-4833-a98b-689f42b18e4c","Type":"ContainerStarted","Data":"378f4e49bf3f431ba2421dc4816c713b6cbbadc7c217cead79b545d77b8b4d82"}
Nov 01 22:22:37.811570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.811535    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-qsqdj-ldvb2"]
Nov 01 22:22:37.815174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.815150    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:37.838959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.838928    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-qsqdj-ldvb2"]
Nov 01 22:22:37.864763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.864736    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"]
Nov 01 22:22:37.867889 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.867874    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:37.873862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.873842    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-6dd789c8b7-ghc6x"]
Nov 01 22:22:37.877220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.877193    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:37.888669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.888647    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Nov 01 22:22:37.888900 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.888880    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Nov 01 22:22:37.891274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.891254    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j4cm5\""
Nov 01 22:22:37.891349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.891317    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Nov 01 22:22:37.891391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.891378    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"downloads-dockercfg-flc7v\""
Nov 01 22:22:37.891473 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.891461    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"]
Nov 01 22:22:37.892349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.892329    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-586fc54c5f-9zzlg"]
Nov 01 22:22:37.897367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.897348    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-ghc6x"]
Nov 01 22:22:37.915762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.915738    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5977\" (UniqueName: \"kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:37.915861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.915785    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:37.915861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.915804    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:37.915954 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.915917    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:37.969188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.969161    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-79448f7f6f-r8bjp"]
Nov 01 22:22:37.972735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.972716    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:37.983436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.983412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:37.988064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:37.988044    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79448f7f6f-r8bjp"]
Nov 01 22:22:38.016264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016241    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp2wb\" (UniqueName: \"kubernetes.io/projected/08dc2d62-7a41-408a-a1cf-0b291f633f2f-kube-api-access-wp2wb\") pod \"downloads-6dd789c8b7-ghc6x\" (UID: \"08dc2d62-7a41-408a-a1cf-0b291f633f2f\") " pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.016383 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016292    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.016435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016408    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5977\" (UniqueName: \"kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.016475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016433    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08dc2d62-7a41-408a-a1cf-0b291f633f2f-tmp\") pod \"downloads-6dd789c8b7-ghc6x\" (UID: \"08dc2d62-7a41-408a-a1cf-0b291f633f2f\") " pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.016475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016458    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8847c4ae-f454-4ec5-99dd-3279b6753bc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-tc5r8\" (UID: \"8847c4ae-f454-4ec5-99dd-3279b6753bc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:38.016604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016544    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.016653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016595    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.016834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.016816    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.017090 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.017071    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.017486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.017464    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.031175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.031154    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5977\" (UniqueName: \"kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977\") pod \"periodic-gathering-qsqdj-ldvb2\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") " pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.117522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117446    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08dc2d62-7a41-408a-a1cf-0b291f633f2f-tmp\") pod \"downloads-6dd789c8b7-ghc6x\" (UID: \"08dc2d62-7a41-408a-a1cf-0b291f633f2f\") " pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.117522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117482    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8847c4ae-f454-4ec5-99dd-3279b6753bc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-tc5r8\" (UID: \"8847c4ae-f454-4ec5-99dd-3279b6753bc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:38.117665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117612    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d8cd38e-474f-47aa-ba46-fe09d424bf53-registry-certificates\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.117665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117654    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg6fq\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-kube-api-access-kg6fq\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.117783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117724    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-registry-tls\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.117837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117797    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d8cd38e-474f-47aa-ba46-fe09d424bf53-trusted-ca\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.117837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117822    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-bound-sa-token\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.117931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117858    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08dc2d62-7a41-408a-a1cf-0b291f633f2f-tmp\") pod \"downloads-6dd789c8b7-ghc6x\" (UID: \"08dc2d62-7a41-408a-a1cf-0b291f633f2f\") " pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.117931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117901    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d8cd38e-474f-47aa-ba46-fe09d424bf53-image-registry-private-configuration\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.117931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117926    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp2wb\" (UniqueName: \"kubernetes.io/projected/08dc2d62-7a41-408a-a1cf-0b291f633f2f-kube-api-access-wp2wb\") pod \"downloads-6dd789c8b7-ghc6x\" (UID: \"08dc2d62-7a41-408a-a1cf-0b291f633f2f\") " pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.118053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.117948    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d8cd38e-474f-47aa-ba46-fe09d424bf53-installation-pull-secrets\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.118117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.118100    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d8cd38e-474f-47aa-ba46-fe09d424bf53-ca-trust-extracted\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.119846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.119830    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/8847c4ae-f454-4ec5-99dd-3279b6753bc5-tls-certificates\") pod \"prometheus-operator-admission-webhook-6489d8bf89-tc5r8\" (UID: \"8847c4ae-f454-4ec5-99dd-3279b6753bc5\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:38.128646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.128629    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:22:38.139370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.139352    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp2wb\" (UniqueName: \"kubernetes.io/projected/08dc2d62-7a41-408a-a1cf-0b291f633f2f-kube-api-access-wp2wb\") pod \"downloads-6dd789c8b7-ghc6x\" (UID: \"08dc2d62-7a41-408a-a1cf-0b291f633f2f\") " pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.180686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.180661    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:38.188405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.188210    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:38.197360 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.197292    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:38.219223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219196    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d8cd38e-474f-47aa-ba46-fe09d424bf53-registry-certificates\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219242    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg6fq\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-kube-api-access-kg6fq\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219269    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-registry-tls\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219316    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d8cd38e-474f-47aa-ba46-fe09d424bf53-trusted-ca\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219339    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-bound-sa-token\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219371    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d8cd38e-474f-47aa-ba46-fe09d424bf53-image-registry-private-configuration\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219405    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d8cd38e-474f-47aa-ba46-fe09d424bf53-installation-pull-secrets\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219436    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d8cd38e-474f-47aa-ba46-fe09d424bf53-ca-trust-extracted\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.219875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.219833    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8d8cd38e-474f-47aa-ba46-fe09d424bf53-ca-trust-extracted\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.220081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.220011    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8d8cd38e-474f-47aa-ba46-fe09d424bf53-registry-certificates\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.220971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.220945    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8d8cd38e-474f-47aa-ba46-fe09d424bf53-trusted-ca\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.222745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.222723    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8d8cd38e-474f-47aa-ba46-fe09d424bf53-image-registry-private-configuration\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.222962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.222940    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8d8cd38e-474f-47aa-ba46-fe09d424bf53-installation-pull-secrets\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.223167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.223146    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-registry-tls\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.233306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.233283    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-bound-sa-token\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.233654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.233605    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg6fq\" (UniqueName: \"kubernetes.io/projected/8d8cd38e-474f-47aa-ba46-fe09d424bf53-kube-api-access-kg6fq\") pod \"image-registry-79448f7f6f-r8bjp\" (UID: \"8d8cd38e-474f-47aa-ba46-fe09d424bf53\") " pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.284728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.284706    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:38.399144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.399117    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-qsqdj-ldvb2"]
Nov 01 22:22:38.401693 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:38.401664    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3977575c_5b80_4a00_b1bb_310cb902b2be.slice/crio-a93901cad1f670b1fdf0bfa7b1630abed2483aebf3619cbdc5e312315ba4e56f WatchSource:0}: Error finding container a93901cad1f670b1fdf0bfa7b1630abed2483aebf3619cbdc5e312315ba4e56f: Status 404 returned error can't find the container with id a93901cad1f670b1fdf0bfa7b1630abed2483aebf3619cbdc5e312315ba4e56f
Nov 01 22:22:38.462452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.462427    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"]
Nov 01 22:22:38.464697 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:38.464673    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8847c4ae_f454_4ec5_99dd_3279b6753bc5.slice/crio-71bcc4d396ce6189023c0e5e5d877cd8c27d28d3167a7e87cc6370a75b99cc15 WatchSource:0}: Error finding container 71bcc4d396ce6189023c0e5e5d877cd8c27d28d3167a7e87cc6370a75b99cc15: Status 404 returned error can't find the container with id 71bcc4d396ce6189023c0e5e5d877cd8c27d28d3167a7e87cc6370a75b99cc15
Nov 01 22:22:38.468806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.468783    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-6dd789c8b7-ghc6x"]
Nov 01 22:22:38.470327 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:38.470303    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08dc2d62_7a41_408a_a1cf_0b291f633f2f.slice/crio-2970e61ef599d6c9ebb42e3d187adeb2631fb4136e075a34f5d68e3353d63e99 WatchSource:0}: Error finding container 2970e61ef599d6c9ebb42e3d187adeb2631fb4136e075a34f5d68e3353d63e99: Status 404 returned error can't find the container with id 2970e61ef599d6c9ebb42e3d187adeb2631fb4136e075a34f5d68e3353d63e99
Nov 01 22:22:38.512293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.512263    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-fzbm2"
Nov 01 22:22:38.558265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.558242    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-79448f7f6f-r8bjp"]
Nov 01 22:22:38.561218 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:38.561186    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8cd38e_474f_47aa_ba46_fe09d424bf53.slice/crio-5d6bd42f310d2eef33c65eac0390fe9d3708666a0226bf4c6fc6ec7e94fba289 WatchSource:0}: Error finding container 5d6bd42f310d2eef33c65eac0390fe9d3708666a0226bf4c6fc6ec7e94fba289: Status 404 returned error can't find the container with id 5d6bd42f310d2eef33c65eac0390fe9d3708666a0226bf4c6fc6ec7e94fba289
Nov 01 22:22:38.581550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.581355    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8" event={"ID":"8847c4ae-f454-4ec5-99dd-3279b6753bc5","Type":"ContainerStarted","Data":"71bcc4d396ce6189023c0e5e5d877cd8c27d28d3167a7e87cc6370a75b99cc15"}
Nov 01 22:22:38.582883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.582857    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-ghc6x" event={"ID":"08dc2d62-7a41-408a-a1cf-0b291f633f2f","Type":"ContainerStarted","Data":"2970e61ef599d6c9ebb42e3d187adeb2631fb4136e075a34f5d68e3353d63e99"}
Nov 01 22:22:38.584828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.584729    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2" event={"ID":"3977575c-5b80-4a00-b1bb-310cb902b2be","Type":"ContainerStarted","Data":"7aa16a078a1627682975bd14a9042cd9e4983a874e5400b123166bf21c28d561"}
Nov 01 22:22:38.584828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.584773    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2" event={"ID":"3977575c-5b80-4a00-b1bb-310cb902b2be","Type":"ContainerStarted","Data":"a93901cad1f670b1fdf0bfa7b1630abed2483aebf3619cbdc5e312315ba4e56f"}
Nov 01 22:22:38.599759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:38.599703    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2" podStartSLOduration=1.599687019 podStartE2EDuration="1.599687019s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:38.59875906 +0000 UTC m=+15.340454679" watchObservedRunningTime="2026-11-01 22:22:38.599687019 +0000 UTC m=+15.341382637"
Nov 01 22:22:39.342240 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.342168    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fzbm2_81511c2e-f1cb-40ea-8d52-b795feb55690/dns/0.log"
Nov 01 22:22:39.543165 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.543136    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fzbm2_81511c2e-f1cb-40ea-8d52-b795feb55690/kube-rbac-proxy/0.log"
Nov 01 22:22:39.592610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.592520    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp" event={"ID":"8d8cd38e-474f-47aa-ba46-fe09d424bf53","Type":"ContainerStarted","Data":"fad4da0c5bacd5630ca7ff0be03deaa9d080d3fdf796a964e7190e8a96b058eb"}
Nov 01 22:22:39.592610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.592584    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp" event={"ID":"8d8cd38e-474f-47aa-ba46-fe09d424bf53","Type":"ContainerStarted","Data":"5d6bd42f310d2eef33c65eac0390fe9d3708666a0226bf4c6fc6ec7e94fba289"}
Nov 01 22:22:39.592804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.592741    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:22:39.617105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.617051    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp" podStartSLOduration=2.6170350940000002 podStartE2EDuration="2.617035094s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:39.615173607 +0000 UTC m=+16.356869227" watchObservedRunningTime="2026-11-01 22:22:39.617035094 +0000 UTC m=+16.358730715"
Nov 01 22:22:39.942613 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:39.942585    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9xfht_295d3d68-c2d1-497f-9a6b-7bd5017b2a41/dns-node-resolver/0.log"
Nov 01 22:22:40.018303 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:22:40.018268    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:22:40.343543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.343487    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-586fc54c5f-9zzlg_aed985ae-387f-4126-a690-6b42829acc8a/registry/0.log"
Nov 01 22:22:40.543595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.543565    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79448f7f6f-r8bjp_8d8cd38e-474f-47aa-ba46-fe09d424bf53/registry/0.log"
Nov 01 22:22:40.598305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.598222    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8" event={"ID":"8847c4ae-f454-4ec5-99dd-3279b6753bc5","Type":"ContainerStarted","Data":"28db9cc7088188a86288690f865d0cfb260f7ea0beef49258bbbc7282189819f"}
Nov 01 22:22:40.598536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.598510    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:40.603643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.603621    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8"
Nov 01 22:22:40.616008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.615968    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-6489d8bf89-tc5r8" podStartSLOduration=2.250171203 podStartE2EDuration="3.615956048s" totalImagesPullingTime="1.365784845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:40.614741676 +0000 UTC m=+17.356437295" watchObservedRunningTime="2026-11-01 22:22:40.615956048 +0000 UTC m=+17.357651682"
Nov 01 22:22:40.743599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:40.743572    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-dfcdddc4b-szq25_0895864a-960a-4fef-9f7e-8d56abeabbcd/registry/0.log"
Nov 01 22:22:41.142369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.142336    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjxrr_4e7fb00c-4f60-4b4c-9402-65cc6ec7e887/node-ca/0.log"
Nov 01 22:22:41.543290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.543236    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8444bb9d4c-m4lpd_ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/router/0.log"
Nov 01 22:22:41.596971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.596926    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:41.697408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.697375    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:41.746185 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.746095    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j85jx_cc378d17-492d-4255-bd20-998d518e8b04/serve-healthcheck-canary/0.log"
Nov 01 22:22:41.815489 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.815373    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:41.919274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:41.919237    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:42.299233 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:42.299185    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:42.408776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:42.408737    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:44.127358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:44.127322    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:44.337284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:44.337250    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:44.697720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:44.697685    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:44.822620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:44.822580    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:46.754453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.754411    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:46.888923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.888847    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:46.958962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.958932    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-sb596"]
Nov 01 22:22:46.965600 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.965572    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:46.970034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.969890    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Nov 01 22:22:46.970223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.970197    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Nov 01 22:22:46.975614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.970560    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-zj5nt\""
Nov 01 22:22:46.975614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.970664    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Nov 01 22:22:46.975614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:46.970731    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Nov 01 22:22:47.019290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.019039    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.104792    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tx9m\" (UniqueName: \"kubernetes.io/projected/ed7e9b70-7315-4c81-8073-d5285edb29d7-kube-api-access-2tx9m\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.104845    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-tls\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.104941    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-root\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.104968    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-sys\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.104994    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.105045    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-textfile\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.105070    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7e9b70-7315-4c81-8073-d5285edb29d7-metrics-client-ca\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.105111    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.105292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.105132    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-wtmp\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.156527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.156478    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:47.205621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205588    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-sys\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.205802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205636    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.205802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205692    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-textfile\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.205802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205719    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7e9b70-7315-4c81-8073-d5285edb29d7-metrics-client-ca\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.205802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205704    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-sys\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.205802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205766    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.205802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205797    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-wtmp\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205843    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tx9m\" (UniqueName: \"kubernetes.io/projected/ed7e9b70-7315-4c81-8073-d5285edb29d7-kube-api-access-2tx9m\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205872    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-tls\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205905    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-root\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.205985    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-root\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.206088    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-textfile\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.206602    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-wtmp\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.206772 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.206723    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ed7e9b70-7315-4c81-8073-d5285edb29d7-metrics-client-ca\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.207171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.207147    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-accelerators-collector-config\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.208507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.208470    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-tls\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.208608 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.208511    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ed7e9b70-7315-4c81-8073-d5285edb29d7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.214347 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.214323    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tx9m\" (UniqueName: \"kubernetes.io/projected/ed7e9b70-7315-4c81-8073-d5285edb29d7-kube-api-access-2tx9m\") pod \"node-exporter-sb596\" (UID: \"ed7e9b70-7315-4c81-8073-d5285edb29d7\") " pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.274691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.274609    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:47.291552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.291524    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-sb596"
Nov 01 22:22:47.307611 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:22:47.307577    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7e9b70_7315_4c81_8073_d5285edb29d7.slice/crio-1fffbecdec240b7efd545c0a037f55536e0428c6390a7854986334cb177e41ea WatchSource:0}: Error finding container 1fffbecdec240b7efd545c0a037f55536e0428c6390a7854986334cb177e41ea: Status 404 returned error can't find the container with id 1fffbecdec240b7efd545c0a037f55536e0428c6390a7854986334cb177e41ea
Nov 01 22:22:47.381848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.381810    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:47.497457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.497424    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:47.632367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.632284    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb596" event={"ID":"ed7e9b70-7315-4c81-8073-d5285edb29d7","Type":"ContainerStarted","Data":"1fffbecdec240b7efd545c0a037f55536e0428c6390a7854986334cb177e41ea"}
Nov 01 22:22:47.899448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.899371    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:22:47.969371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:47.969335    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:48.090079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:48.090044    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:48.250333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:48.250244    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:48.641141 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:48.641107    5057 generic.go:356] "Generic (PLEG): container finished" podID="ed7e9b70-7315-4c81-8073-d5285edb29d7" containerID="09a02f1f022c2d3018688b0c92d5b36d6701a45c0a565e0815be970dd7148adc" exitCode=0
Nov 01 22:22:48.641327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:48.641166    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb596" event={"ID":"ed7e9b70-7315-4c81-8073-d5285edb29d7","Type":"ContainerDied","Data":"09a02f1f022c2d3018688b0c92d5b36d6701a45c0a565e0815be970dd7148adc"}
Nov 01 22:22:49.648988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:49.648955    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb596" event={"ID":"ed7e9b70-7315-4c81-8073-d5285edb29d7","Type":"ContainerStarted","Data":"c8bd5f9bb7eb48353e554d737fc9012a5dcb5c2c22dceb6817c941aa6dfd97a6"}
Nov 01 22:22:49.649412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:49.649006    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-sb596" event={"ID":"ed7e9b70-7315-4c81-8073-d5285edb29d7","Type":"ContainerStarted","Data":"a886a6e895c0754edffdd3fa53d65b68d97ee984edf1d5285d285afa46a06f30"}
Nov 01 22:22:49.670439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:49.670378    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-sb596" podStartSLOduration=2.435803364 podStartE2EDuration="3.670360335s" totalImagesPullingTime="1.160038889s" totalInitContainerRuntime="74.518082ms" isStatefulPod=true podCreationTimestamp="2026-11-01 22:22:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:49.668225146 +0000 UTC m=+26.409920765" watchObservedRunningTime="2026-11-01 22:22:49.670360335 +0000 UTC m=+26.412055952"
Nov 01 22:22:49.942777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:49.942695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:50.051087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:50.051050    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:50.157684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:50.157644    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:50.717320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:50.717287    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:50.830958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:50.830916    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:51.527255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:51.527218    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:22:51.748000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:51.747958    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:51.858509 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:51.858410    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:51.976630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:51.976586    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:53.265199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:53.265163    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:53.832437 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:53.832406    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5gq2"
Nov 01 22:22:56.869092 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:56.869060    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:57.009649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:57.009613    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:22:57.693043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:57.692970    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-6dd789c8b7-ghc6x" event={"ID":"08dc2d62-7a41-408a-a1cf-0b291f633f2f","Type":"ContainerStarted","Data":"2fa400e35b6d985dda143119eadf87e91a9d1aec9bc53357143a98807349d61a"}
Nov 01 22:22:57.693308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:57.693144    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:22:57.695108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:57.695081    5057 patch_prober.go:29] interesting pod/downloads-6dd789c8b7-ghc6x container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.30:8080/\": dial tcp 10.132.0.30:8080: connect: connection refused" start-of-body=
Nov 01 22:22:57.695245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:57.695138    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-6dd789c8b7-ghc6x" podUID="08dc2d62-7a41-408a-a1cf-0b291f633f2f" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.30:8080/\": dial tcp 10.132.0.30:8080: connect: connection refused"
Nov 01 22:22:57.715654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:57.714606    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/downloads-6dd789c8b7-ghc6x" podStartSLOduration=1.868707299 podStartE2EDuration="20.714588652s" totalImagesPullingTime="18.845881353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:22:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:22:57.714000968 +0000 UTC m=+34.455696588" watchObservedRunningTime="2026-11-01 22:22:57.714588652 +0000 UTC m=+34.456284271"
Nov 01 22:22:58.405797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:58.405764    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-lg7nw"
Nov 01 22:22:58.708770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:22:58.708735    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-6dd789c8b7-ghc6x"
Nov 01 22:23:00.201707 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:00.201657    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:00.309183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:00.309145    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:00.602927 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:00.602850    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-79448f7f6f-r8bjp"
Nov 01 22:23:00.652527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:00.650204    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dfcdddc4b-szq25"]
Nov 01 22:23:00.774862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:00.774825    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:00.877514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:00.877380    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:01.075069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:01.074985    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:01.188609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:01.188529    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:02.175814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:02.175775    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:02.293752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:02.293717    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:02.915339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:02.915245    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg" podUID="aed985ae-387f-4126-a690-6b42829acc8a" containerName="registry" containerID="cri-o://cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8" gracePeriod=30
Nov 01 22:23:03.142124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.142095    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:03.212200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.212168    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:23:03.267677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267644    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267697    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267740    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267806    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267847    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267881    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267917    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.267997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.267970    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") pod \"aed985ae-387f-4126-a690-6b42829acc8a\" (UID: \"aed985ae-387f-4126-a690-6b42829acc8a\") "
Nov 01 22:23:03.268183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.268132    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:23:03.268275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.268257    5057 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-trusted-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.271136 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.271102    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:23:03.271366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.271102    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:03.271829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.271786    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "kube-api-access-w2sbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:03.272348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.272321    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:23:03.273296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.273265    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:23:03.274388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.274367    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:03.278156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.278126    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted" pod "aed985ae-387f-4126-a690-6b42829acc8a" (UID: "aed985ae-387f-4126-a690-6b42829acc8a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:23:03.321282 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.321250    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:03.369327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369299    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2sbv\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-kube-api-access-w2sbv\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.369327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369326    5057 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/aed985ae-387f-4126-a690-6b42829acc8a-registry-certificates\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.369481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369341    5057 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-bound-sa-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.369481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369354    5057 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-installation-pull-secrets\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.369481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369367    5057 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/aed985ae-387f-4126-a690-6b42829acc8a-ca-trust-extracted\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.369481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369380    5057 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/aed985ae-387f-4126-a690-6b42829acc8a-registry-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.369481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.369394    5057 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/aed985ae-387f-4126-a690-6b42829acc8a-image-registry-private-configuration\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:03.727770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.727729    5057 generic.go:356] "Generic (PLEG): container finished" podID="aed985ae-387f-4126-a690-6b42829acc8a" containerID="cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8" exitCode=0
Nov 01 22:23:03.728046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.727812    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg"
Nov 01 22:23:03.728046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.727825    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg" event={"ID":"aed985ae-387f-4126-a690-6b42829acc8a","Type":"ContainerDied","Data":"cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8"}
Nov 01 22:23:03.728046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.727890    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-586fc54c5f-9zzlg" event={"ID":"aed985ae-387f-4126-a690-6b42829acc8a","Type":"ContainerDied","Data":"8bd5b472937334253757e8c4eec061fef95447fca115820396260cba906a9289"}
Nov 01 22:23:03.728046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.727921    5057 scope.go:122] "RemoveContainer" containerID="cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8"
Nov 01 22:23:03.738561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.738540    5057 scope.go:122] "RemoveContainer" containerID="cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8"
Nov 01 22:23:03.738895 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:23:03.738866    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8\": container with ID starting with cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8 not found: ID does not exist" containerID="cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8"
Nov 01 22:23:03.738991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.738901    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8"} err="failed to get container status \"cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8\": rpc error: code = NotFound desc = could not find container \"cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8\": container with ID starting with cc230ed891bd23bdccace468b8f9edcbf39846008296ad7bad68902dfb3c77d8 not found: ID does not exist"
Nov 01 22:23:03.746564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.746541    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-586fc54c5f-9zzlg"]
Nov 01 22:23:03.750965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:03.750941    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-586fc54c5f-9zzlg"]
Nov 01 22:23:05.396050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:05.396017    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="aed985ae-387f-4126-a690-6b42829acc8a" path="/var/lib/kubelet/pods/aed985ae-387f-4126-a690-6b42829acc8a/volumes"
Nov 01 22:23:10.012998 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:23:10.012963    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:23:10.873096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:10.873061    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-75nhs-z7sqb"]
Nov 01 22:23:10.873337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:10.873297    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-75nhs-z7sqb" podUID="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" containerName="insights-gathering" containerID="cri-o://f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553" gracePeriod=30
Nov 01 22:23:11.210653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.210627    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:23:11.336795 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.336762    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") pod \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") "
Nov 01 22:23:11.336993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.336818    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") pod \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") "
Nov 01 22:23:11.336993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.336857    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") pod \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") "
Nov 01 22:23:11.336993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.336903    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") pod \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\" (UID: \"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5\") "
Nov 01 22:23:11.337227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.337198    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path" pod "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" (UID: "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:23:11.337302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.337206    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle" pod "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" (UID: "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:23:11.337302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.337283    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle" pod "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" (UID: "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:23:11.339352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.339318    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:11.339607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.339585    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts" pod "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" (UID: "12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5"). InnerVolumeSpecName "kube-api-access-cznts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:11.438413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.438339    5057 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-trusted-ca-bundle\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:11.438413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.438369    5057 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-archives-path\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:11.438413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.438379    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cznts\" (UniqueName: \"kubernetes.io/projected/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-kube-api-access-cznts\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:11.438413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.438390    5057 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5-service-ca-bundle\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:11.459411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.459373    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:11.765357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.765319    5057 generic.go:356] "Generic (PLEG): container finished" podID="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" containerID="f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553" exitCode=2
Nov 01 22:23:11.765567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.765383    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-75nhs-z7sqb"
Nov 01 22:23:11.765567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.765400    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75nhs-z7sqb" event={"ID":"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5","Type":"ContainerDied","Data":"f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553"}
Nov 01 22:23:11.765567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.765453    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-75nhs-z7sqb" event={"ID":"12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5","Type":"ContainerDied","Data":"a879dc931a0686c42715756f8d27b7217af9d0b7ccd53a26b1436f931d894c29"}
Nov 01 22:23:11.765567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.765474    5057 scope.go:122] "RemoveContainer" containerID="f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553"
Nov 01 22:23:11.773660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.773644    5057 scope.go:122] "RemoveContainer" containerID="f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553"
Nov 01 22:23:11.773909 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:23:11.773891    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553\": container with ID starting with f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553 not found: ID does not exist" containerID="f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553"
Nov 01 22:23:11.773955 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.773916    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553"} err="failed to get container status \"f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553\": rpc error: code = NotFound desc = could not find container \"f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553\": container with ID starting with f50720c4f967a88e298fa5d9e090263a0f143737d6229cdaab18ed21ff579553 not found: ID does not exist"
Nov 01 22:23:11.787316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.787290    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-75nhs-z7sqb"]
Nov 01 22:23:11.790905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.790878    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-75nhs-z7sqb"]
Nov 01 22:23:11.824560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.824527    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:11.930862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:11.930829    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:12.076232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:12.076148    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:12.215362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:12.215325    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:12.582639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:12.582586    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:12.701010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:12.700978    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:13.395002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:13.394967    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" path="/var/lib/kubelet/pods/12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5/volumes"
Nov 01 22:23:18.792159 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:18.792116    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:18.904167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:18.904124    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:19.088096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:19.088008    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:22.570249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:22.570211    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:23.238761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:23.238722    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:23.378101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:23.378070    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:25.485217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:25.485174    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:25.620797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:25.620759    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:25.677044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:25.677002    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25" podUID="0895864a-960a-4fef-9f7e-8d56abeabbcd" containerName="registry" containerID="cri-o://1bcae5c2738020ee18db7de3a22eb7a5bcec050cad20cd9760c7ec83ea7b01d9" gracePeriod=30
Nov 01 22:23:25.827155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:25.827118    5057 generic.go:356] "Generic (PLEG): container finished" podID="0895864a-960a-4fef-9f7e-8d56abeabbcd" containerID="1bcae5c2738020ee18db7de3a22eb7a5bcec050cad20cd9760c7ec83ea7b01d9" exitCode=0
Nov 01 22:23:25.827284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:25.827207    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25" event={"ID":"0895864a-960a-4fef-9f7e-8d56abeabbcd","Type":"ContainerDied","Data":"1bcae5c2738020ee18db7de3a22eb7a5bcec050cad20cd9760c7ec83ea7b01d9"}
Nov 01 22:23:25.958912 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:25.958886    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:23:26.060853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.060770    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.060853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.060827    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.060862    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.060893    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.060934    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.061072    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.061153    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.061184    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") pod \"0895864a-960a-4fef-9f7e-8d56abeabbcd\" (UID: \"0895864a-960a-4fef-9f7e-8d56abeabbcd\") "
Nov 01 22:23:26.061404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.061329    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:23:26.061637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.061575    5057 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-certificates\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.061745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.061722    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:23:26.063684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.063656    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:23:26.063812 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.063716    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "kube-api-access-m9sgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:26.063812 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.063801    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:26.063938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.063811    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:23:26.063938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.063848    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:23:26.067837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.067813    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted" pod "0895864a-960a-4fef-9f7e-8d56abeabbcd" (UID: "0895864a-960a-4fef-9f7e-8d56abeabbcd"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:23:26.162892 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162860    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9sgc\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-kube-api-access-m9sgc\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.162892 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162890    5057 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-image-registry-private-configuration\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.163064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162902    5057 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-registry-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.163064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162911    5057 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0895864a-960a-4fef-9f7e-8d56abeabbcd-ca-trust-extracted\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.163064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162920    5057 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0895864a-960a-4fef-9f7e-8d56abeabbcd-bound-sa-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.163064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162929    5057 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0895864a-960a-4fef-9f7e-8d56abeabbcd-installation-pull-secrets\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.163064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.162937    5057 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0895864a-960a-4fef-9f7e-8d56abeabbcd-trusted-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:23:26.833150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.833115    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25" event={"ID":"0895864a-960a-4fef-9f7e-8d56abeabbcd","Type":"ContainerDied","Data":"ca6f4fc7b0463d0012c8a52213b6e57093eb4dee11654bccc09577a12f862193"}
Nov 01 22:23:26.833564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.833171    5057 scope.go:122] "RemoveContainer" containerID="1bcae5c2738020ee18db7de3a22eb7a5bcec050cad20cd9760c7ec83ea7b01d9"
Nov 01 22:23:26.833564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.833126    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-dfcdddc4b-szq25"
Nov 01 22:23:26.855448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.855422    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-dfcdddc4b-szq25"]
Nov 01 22:23:26.858951 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.858932    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-dfcdddc4b-szq25"]
Nov 01 22:23:26.975894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:26.975862    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:27.105941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:27.105854    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:27.394358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:27.394291    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0895864a-960a-4fef-9f7e-8d56abeabbcd" path="/var/lib/kubelet/pods/0895864a-960a-4fef-9f7e-8d56abeabbcd/volumes"
Nov 01 22:23:28.441568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:28.441532    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:28.567600 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:28.567564    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:28.836538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:28.836464    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:28.957926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:28.957893    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:40.012776 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:23:40.012741    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:23:41.836890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:41.836848    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:41.957271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:41.957235    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:23:53.832714 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:23:53.832680    5057 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5dfc1c_8c00_45aa_b941_64f36a914db3.slice/crio-conmon-dc4b9f883f249a6fdc085e9385c10b9146cf8cfa6c53d921793a206e81aec529.scope\": RecentStats: unable to find data in memory cache]"
Nov 01 22:23:53.945380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:53.945341    5057 generic.go:356] "Generic (PLEG): container finished" podID="4b5dfc1c-8c00-45aa-b941-64f36a914db3" containerID="dc4b9f883f249a6fdc085e9385c10b9146cf8cfa6c53d921793a206e81aec529" exitCode=0
Nov 01 22:23:53.945576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:53.945412    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8" event={"ID":"4b5dfc1c-8c00-45aa-b941-64f36a914db3","Type":"ContainerDied","Data":"dc4b9f883f249a6fdc085e9385c10b9146cf8cfa6c53d921793a206e81aec529"}
Nov 01 22:23:53.945772 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:53.945758    5057 scope.go:122] "RemoveContainer" containerID="dc4b9f883f249a6fdc085e9385c10b9146cf8cfa6c53d921793a206e81aec529"
Nov 01 22:23:54.951368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:23:54.951334    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-q2lj8" event={"ID":"4b5dfc1c-8c00-45aa-b941-64f36a914db3","Type":"ContainerStarted","Data":"f385d5ae445e12a46c7fefc0d5074278c1732138428407fcfe8c20cf18d45e79"}
Nov 01 22:24:03.987099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:03.987061    5057 generic.go:356] "Generic (PLEG): container finished" podID="d91a8683-96c3-4725-b90b-da6590c5076e" containerID="826c0cd4bc000729604d8e7efae9b298400a59b85a23aa756217839aea8bfe21" exitCode=0
Nov 01 22:24:03.987556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:03.987118    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" event={"ID":"d91a8683-96c3-4725-b90b-da6590c5076e","Type":"ContainerDied","Data":"826c0cd4bc000729604d8e7efae9b298400a59b85a23aa756217839aea8bfe21"}
Nov 01 22:24:03.987556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:03.987165    5057 scope.go:122] "RemoveContainer" containerID="4584d47bd57acc759d2c58db24b08b4fbb0e8061e6c64c16ba773dd0460a445b"
Nov 01 22:24:03.987556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:03.987470    5057 scope.go:122] "RemoveContainer" containerID="826c0cd4bc000729604d8e7efae9b298400a59b85a23aa756217839aea8bfe21"
Nov 01 22:24:03.987734 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:24:03.987711    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"insights-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=insights-operator pod=insights-operator-7c4986d449-fvxj8_openshift-insights(d91a8683-96c3-4725-b90b-da6590c5076e)\"" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" podUID="d91a8683-96c3-4725-b90b-da6590c5076e"
Nov 01 22:24:04.992775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:04.992743    5057 generic.go:356] "Generic (PLEG): container finished" podID="3977575c-5b80-4a00-b1bb-310cb902b2be" containerID="7aa16a078a1627682975bd14a9042cd9e4983a874e5400b123166bf21c28d561" exitCode=0
Nov 01 22:24:04.993187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:04.992818    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2" event={"ID":"3977575c-5b80-4a00-b1bb-310cb902b2be","Type":"ContainerDied","Data":"7aa16a078a1627682975bd14a9042cd9e4983a874e5400b123166bf21c28d561"}
Nov 01 22:24:06.135469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.135445    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:24:06.164454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164423    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle\") pod \"3977575c-5b80-4a00-b1bb-310cb902b2be\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") "
Nov 01 22:24:06.164623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164475    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle\") pod \"3977575c-5b80-4a00-b1bb-310cb902b2be\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") "
Nov 01 22:24:06.164623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164530    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path\") pod \"3977575c-5b80-4a00-b1bb-310cb902b2be\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") "
Nov 01 22:24:06.164623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164580    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977\" (UniqueName: \"kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977\") pod \"3977575c-5b80-4a00-b1bb-310cb902b2be\" (UID: \"3977575c-5b80-4a00-b1bb-310cb902b2be\") "
Nov 01 22:24:06.164842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164818    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path" pod "3977575c-5b80-4a00-b1bb-310cb902b2be" (UID: "3977575c-5b80-4a00-b1bb-310cb902b2be"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:24:06.164903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164846    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle" pod "3977575c-5b80-4a00-b1bb-310cb902b2be" (UID: "3977575c-5b80-4a00-b1bb-310cb902b2be"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:24:06.164903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164856    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle" pod "3977575c-5b80-4a00-b1bb-310cb902b2be" (UID: "3977575c-5b80-4a00-b1bb-310cb902b2be"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:24:06.164903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.164888    5057 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/3977575c-5b80-4a00-b1bb-310cb902b2be-archives-path\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:24:06.166739 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.166711    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977" pod "3977575c-5b80-4a00-b1bb-310cb902b2be" (UID: "3977575c-5b80-4a00-b1bb-310cb902b2be"). InnerVolumeSpecName "kube-api-access-p5977". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:24:06.265701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.265626    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5977\" (UniqueName: \"kubernetes.io/projected/3977575c-5b80-4a00-b1bb-310cb902b2be-kube-api-access-p5977\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:24:06.265701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.265655    5057 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-trusted-ca-bundle\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:24:06.265701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:06.265664    5057 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3977575c-5b80-4a00-b1bb-310cb902b2be-service-ca-bundle\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:24:07.003861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:07.003823    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2" event={"ID":"3977575c-5b80-4a00-b1bb-310cb902b2be","Type":"ContainerDied","Data":"a93901cad1f670b1fdf0bfa7b1630abed2483aebf3619cbdc5e312315ba4e56f"}
Nov 01 22:24:07.004049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:07.003876    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a93901cad1f670b1fdf0bfa7b1630abed2483aebf3619cbdc5e312315ba4e56f"
Nov 01 22:24:07.004049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:07.003841    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-qsqdj-ldvb2"
Nov 01 22:24:10.011054 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:24:10.011016    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:24:12.114746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:12.114711    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:24:12.241657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:12.241615    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:24:14.390353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:14.390269    5057 scope.go:122] "RemoveContainer" containerID="826c0cd4bc000729604d8e7efae9b298400a59b85a23aa756217839aea8bfe21"
Nov 01 22:24:15.037096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:15.037062    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-fvxj8" event={"ID":"d91a8683-96c3-4725-b90b-da6590c5076e","Type":"ContainerStarted","Data":"a212b7166659aa34a087724d7051b564e100f46e2dc5ffaf15f3db076c2c4ece"}
Nov 01 22:24:28.463479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:24:28.463441    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:24:40.013540 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:24:40.013445    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:25:10.013089 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:25:10.013049    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:25:12.143053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:12.143011    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:30.405079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:30.404981    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:30.551954 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:30.551916    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:31.735792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:31.735761    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:31.860457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:31.860421    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:32.635069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:32.635034    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:36.908731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:36.908693    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:37.021643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.021603    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:37.221686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.221652    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"]
Nov 01 22:25:37.221980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.221967    5057 cpu_manager.go:414] "containerMap: removing container" podUID="aed985ae-387f-4126-a690-6b42829acc8a" containerName="registry"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.221980    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="aed985ae-387f-4126-a690-6b42829acc8a" containerName="registry"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.221996    5057 cpu_manager.go:414] "containerMap: removing container" podUID="0895864a-960a-4fef-9f7e-8d56abeabbcd" containerName="registry"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222001    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0895864a-960a-4fef-9f7e-8d56abeabbcd" containerName="registry"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222010    5057 cpu_manager.go:414] "containerMap: removing container" podUID="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" containerName="insights-gathering"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222015    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" containerName="insights-gathering"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222024    5057 cpu_manager.go:414] "containerMap: removing container" podUID="3977575c-5b80-4a00-b1bb-310cb902b2be" containerName="insights-gathering"
Nov 01 22:25:37.222046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222029    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3977575c-5b80-4a00-b1bb-310cb902b2be" containerName="insights-gathering"
Nov 01 22:25:37.222397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222090    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="aed985ae-387f-4126-a690-6b42829acc8a" containerName="registry"
Nov 01 22:25:37.222397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222098    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="0895864a-960a-4fef-9f7e-8d56abeabbcd" containerName="registry"
Nov 01 22:25:37.222397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222104    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="12bf6db7-4db5-40b6-ae97-bf1f59bdc4a5" containerName="insights-gathering"
Nov 01 22:25:37.222397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.222111    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="3977575c-5b80-4a00-b1bb-310cb902b2be" containerName="insights-gathering"
Nov 01 22:25:37.225113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.225091    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.240523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.238739    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"]
Nov 01 22:25:37.244421 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.244400    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"openshift-ai-inference-openshift-ai-inference-dockercfg-tdb96\""
Nov 01 22:25:37.255450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.255429    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Nov 01 22:25:37.265578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265551    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27e9d85-ba64-411d-9423-b3819825ea02-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265584    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265602    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27e9d85-ba64-411d-9423-b3819825ea02-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265768 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265675    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265768 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265729    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265768 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265758    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27e9d85-ba64-411d-9423-b3819825ea02-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265774    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7lxx\" (UniqueName: \"kubernetes.io/projected/c27e9d85-ba64-411d-9423-b3819825ea02-kube-api-access-p7lxx\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265797    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.265972 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.265921    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.296857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.296831    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:37.366449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366422    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.366641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366453    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.366641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366483    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27e9d85-ba64-411d-9423-b3819825ea02-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.366641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366524    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7lxx\" (UniqueName: \"kubernetes.io/projected/c27e9d85-ba64-411d-9423-b3819825ea02-kube-api-access-p7lxx\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.366641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366552    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.366641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366574    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.366641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366616    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27e9d85-ba64-411d-9423-b3819825ea02-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366643    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.366669    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27e9d85-ba64-411d-9423-b3819825ea02-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367119 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.367070    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-istio-data\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.367129    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-workload-socket\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.367190    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-workload-certs\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.367328    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-credential-socket\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.367412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.367380    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c27e9d85-ba64-411d-9423-b3819825ea02-istiod-ca-cert\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.368834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.368813    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c27e9d85-ba64-411d-9423-b3819825ea02-istio-envoy\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.368970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.368950    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c27e9d85-ba64-411d-9423-b3819825ea02-istio-podinfo\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.374214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.374191    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c27e9d85-ba64-411d-9423-b3819825ea02-istio-token\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.374536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.374519    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7lxx\" (UniqueName: \"kubernetes.io/projected/c27e9d85-ba64-411d-9423-b3819825ea02-kube-api-access-p7lxx\") pod \"openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6\" (UID: \"c27e9d85-ba64-411d-9423-b3819825ea02\") " pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.414547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.414514    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:37.541332 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.541249    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:37.825515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:37.825478    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"]
Nov 01 22:25:37.827927 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:25:37.827900    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc27e9d85_ba64_411d_9423_b3819825ea02.slice/crio-2cb0762ad05f9d6abb74269232e20939290b52ced147a001c70e4a8093f217d3 WatchSource:0}: Error finding container 2cb0762ad05f9d6abb74269232e20939290b52ced147a001c70e4a8093f217d3: Status 404 returned error can't find the container with id 2cb0762ad05f9d6abb74269232e20939290b52ced147a001c70e4a8093f217d3
Nov 01 22:25:38.362628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:38.362591    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6" event={"ID":"c27e9d85-ba64-411d-9423-b3819825ea02","Type":"ContainerStarted","Data":"2cb0762ad05f9d6abb74269232e20939290b52ced147a001c70e4a8093f217d3"}
Nov 01 22:25:39.808249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:39.808207    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:40.013097 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:25:40.013006    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:25:42.505438 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:42.505406    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:25:42.505740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:42.505473    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:25:42.505740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:42.505518    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:25:43.384763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:43.384729    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6" event={"ID":"c27e9d85-ba64-411d-9423-b3819825ea02","Type":"ContainerStarted","Data":"be0dd54e1886245f5630c256f208285e1e65c9804b4abe478ffa91f0932b48f1"}
Nov 01 22:25:43.404085 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:43.404033    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6" podStartSLOduration=1.729516761 podStartE2EDuration="6.40401961s" totalImagesPullingTime="4.674502849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:25:43.403441701 +0000 UTC m=+200.145137321" watchObservedRunningTime="2026-11-01 22:25:43.40401961 +0000 UTC m=+200.145715228"
Nov 01 22:25:43.542013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:43.541975    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:43.547402 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:43.547375    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:44.388790 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:44.388759    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:44.389819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:44.389798    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6"
Nov 01 22:25:46.356036 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:46.356002    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:46.461381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:46.461353    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:49.305708 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:49.305671    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:25:49.416764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:25:49.416730    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:10.012943 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:26:10.012906    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:26:11.886033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:11.885996    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:12.077578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:12.077545    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:12.202013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:12.201978    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:13.893508 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:13.893458    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:17.497412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:17.497376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:18.891255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:18.891217    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:21.102485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:21.102449    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:21.705440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:21.705408    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:22.494014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:22.493970    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:25.103939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:25.103894    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:25.282263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:25.282231    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:26.491008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.490970    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:26.581959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.581928    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"]
Nov 01 22:26:26.590222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.590203    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.597131 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.597107    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Nov 01 22:26:26.597286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.597263    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Nov 01 22:26:26.597398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.597375    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Nov 01 22:26:26.597472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.597395    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Nov 01 22:26:26.598724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.598682    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-fc42r\""
Nov 01 22:26:26.598828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.598750    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Nov 01 22:26:26.613773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.613750    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"]
Nov 01 22:26:26.662354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.662326    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-metrics-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.662524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.662361    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zrp7z\" (UniqueName: \"kubernetes.io/projected/d74ad49a-0cce-4daa-a991-0e65c6c24765-kube-api-access-zrp7z\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.662524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.662382    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-webhook-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.662524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.662444    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d74ad49a-0cce-4daa-a991-0e65c6c24765-tmp\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.662715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.662548    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d74ad49a-0cce-4daa-a991-0e65c6c24765-manifests\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.662715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.662574    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.763633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.763561    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-metrics-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.763633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.763593    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zrp7z\" (UniqueName: \"kubernetes.io/projected/d74ad49a-0cce-4daa-a991-0e65c6c24765-kube-api-access-zrp7z\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.763633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.763616    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-webhook-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.763897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.763733    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d74ad49a-0cce-4daa-a991-0e65c6c24765-tmp\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.763897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.763813    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d74ad49a-0cce-4daa-a991-0e65c6c24765-manifests\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.763897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.763841    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.764268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.764238    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d74ad49a-0cce-4daa-a991-0e65c6c24765-manifests\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.766167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.766138    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d74ad49a-0cce-4daa-a991-0e65c6c24765-tmp\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.766283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.766212    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.766391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.766370    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-metrics-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.766449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.766418    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d74ad49a-0cce-4daa-a991-0e65c6c24765-webhook-cert\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.787142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.787118    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zrp7z\" (UniqueName: \"kubernetes.io/projected/d74ad49a-0cce-4daa-a991-0e65c6c24765-kube-api-access-zrp7z\") pod \"opendatahub-operator-controller-manager-8487c6b6d6-f98qg\" (UID: \"d74ad49a-0cce-4daa-a991-0e65c6c24765\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.834308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.834261    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:26.905680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.905647    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:26.966883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:26.966372    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:27.130923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.130830    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:27.209950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.209924    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"]
Nov 01 22:26:27.212682 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:26:27.212646    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd74ad49a_0cce_4daa_a991_0e65c6c24765.slice/crio-65892abcc18e89aef6b8ce6f40ea9a777d798da74c9de239ba538d6de83aad78 WatchSource:0}: Error finding container 65892abcc18e89aef6b8ce6f40ea9a777d798da74c9de239ba538d6de83aad78: Status 404 returned error can't find the container with id 65892abcc18e89aef6b8ce6f40ea9a777d798da74c9de239ba538d6de83aad78
Nov 01 22:26:27.264044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.264018    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:27.417748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.417657    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:27.560971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.560941    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg" event={"ID":"d74ad49a-0cce-4daa-a991-0e65c6c24765","Type":"ContainerStarted","Data":"65892abcc18e89aef6b8ce6f40ea9a777d798da74c9de239ba538d6de83aad78"}
Nov 01 22:26:27.838629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.838586    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:27.898906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.898868    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"]
Nov 01 22:26:27.902189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.902174    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:27.910725 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.910702    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Nov 01 22:26:27.910725 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.910702    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Nov 01 22:26:27.912674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.912657    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gptd7\""
Nov 01 22:26:27.917079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.917056    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"]
Nov 01 22:26:27.972334 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.972305    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-989th\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:27.972503 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.972360    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44rch\" (UniqueName: \"kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch\") pod \"kuadrant-operator-controller-manager-cf8469d7b-989th\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:27.985390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:27.985339    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:28.073770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.073736    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-989th\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:28.073921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.073780    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44rch\" (UniqueName: \"kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch\") pod \"kuadrant-operator-controller-manager-cf8469d7b-989th\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:28.074145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.074122    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-cf8469d7b-989th\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:28.097435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.097363    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-44rch\" (UniqueName: \"kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch\") pod \"kuadrant-operator-controller-manager-cf8469d7b-989th\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:28.155220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.155187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:28.214907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.214873    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:28.301754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.301689    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:28.487966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.487929    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:28.511545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.511513    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"]
Nov 01 22:26:28.515791 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:26:28.515766    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f45aac_69b5_4da9_b8a2_7208b9236db8.slice/crio-af5c250f24e803584e28bba83e1d3e11c08b79967df17bd8805fef88ec9a18e6 WatchSource:0}: Error finding container af5c250f24e803584e28bba83e1d3e11c08b79967df17bd8805fef88ec9a18e6: Status 404 returned error can't find the container with id af5c250f24e803584e28bba83e1d3e11c08b79967df17bd8805fef88ec9a18e6
Nov 01 22:26:28.566290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:28.566262    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th" event={"ID":"f9f45aac-69b5-4da9-b8a2-7208b9236db8","Type":"ContainerStarted","Data":"af5c250f24e803584e28bba83e1d3e11c08b79967df17bd8805fef88ec9a18e6"}
Nov 01 22:26:29.174163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:29.174130    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:29.315719 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:29.315576    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:29.455477 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:29.455439    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:30.527736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.527700    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:30.842182 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.842082    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:30.899791 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.899749    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"]
Nov 01 22:26:30.904056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.904029    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:30.911787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.911763    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-operator-dockercfg-8n999\""
Nov 01 22:26:30.911911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.911896    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\""
Nov 01 22:26:30.911995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.911980    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\""
Nov 01 22:26:30.912204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:30.912178    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"]
Nov 01 22:26:31.001871 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.001828    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntddr\" (UniqueName: \"kubernetes.io/projected/86b517fe-edbf-4182-803d-f61b65a6eb09-kube-api-access-ntddr\") pod \"jobset-operator-674cd59dc5-jxxbm\" (UID: \"86b517fe-edbf-4182-803d-f61b65a6eb09\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.002055 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.002012    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86b517fe-edbf-4182-803d-f61b65a6eb09-tmp\") pod \"jobset-operator-674cd59dc5-jxxbm\" (UID: \"86b517fe-edbf-4182-803d-f61b65a6eb09\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.102745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.102663    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86b517fe-edbf-4182-803d-f61b65a6eb09-tmp\") pod \"jobset-operator-674cd59dc5-jxxbm\" (UID: \"86b517fe-edbf-4182-803d-f61b65a6eb09\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.102914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.102835    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntddr\" (UniqueName: \"kubernetes.io/projected/86b517fe-edbf-4182-803d-f61b65a6eb09-kube-api-access-ntddr\") pod \"jobset-operator-674cd59dc5-jxxbm\" (UID: \"86b517fe-edbf-4182-803d-f61b65a6eb09\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.103105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.103084    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/86b517fe-edbf-4182-803d-f61b65a6eb09-tmp\") pod \"jobset-operator-674cd59dc5-jxxbm\" (UID: \"86b517fe-edbf-4182-803d-f61b65a6eb09\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.111668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.111644    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntddr\" (UniqueName: \"kubernetes.io/projected/86b517fe-edbf-4182-803d-f61b65a6eb09-kube-api-access-ntddr\") pod \"jobset-operator-674cd59dc5-jxxbm\" (UID: \"86b517fe-edbf-4182-803d-f61b65a6eb09\") " pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.230771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.230747    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"
Nov 01 22:26:31.537417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.537389    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm"]
Nov 01 22:26:31.541048 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:26:31.541016    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86b517fe_edbf_4182_803d_f61b65a6eb09.slice/crio-9dc3c389dfacc2a9b47a0a0a55d757c528a43020e0fc1a9cf35295d9aaf51544 WatchSource:0}: Error finding container 9dc3c389dfacc2a9b47a0a0a55d757c528a43020e0fc1a9cf35295d9aaf51544: Status 404 returned error can't find the container with id 9dc3c389dfacc2a9b47a0a0a55d757c528a43020e0fc1a9cf35295d9aaf51544
Nov 01 22:26:31.585586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.585064    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg" event={"ID":"d74ad49a-0cce-4daa-a991-0e65c6c24765","Type":"ContainerStarted","Data":"776fd5d22ee36baa84eb858501e53336be65e65c6734dc79c543425d37e8148a"}
Nov 01 22:26:31.587030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:31.586993    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm" event={"ID":"86b517fe-edbf-4182-803d-f61b65a6eb09","Type":"ContainerStarted","Data":"9dc3c389dfacc2a9b47a0a0a55d757c528a43020e0fc1a9cf35295d9aaf51544"}
Nov 01 22:26:32.426454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:32.426416    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:32.592650 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:32.592612    5057 generic.go:356] "Generic (PLEG): container finished" podID="d74ad49a-0cce-4daa-a991-0e65c6c24765" containerID="776fd5d22ee36baa84eb858501e53336be65e65c6734dc79c543425d37e8148a" exitCode=0
Nov 01 22:26:32.593088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:32.592696    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg" event={"ID":"d74ad49a-0cce-4daa-a991-0e65c6c24765","Type":"ContainerDied","Data":"776fd5d22ee36baa84eb858501e53336be65e65c6734dc79c543425d37e8148a"}
Nov 01 22:26:33.124933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.124896    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:33.272353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.272180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:33.297893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.297864    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"]
Nov 01 22:26:33.301550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.301522    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:33.309085 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.309046    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-rzhgw\""
Nov 01 22:26:33.310363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.310315    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"]
Nov 01 22:26:33.331295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.331225    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t72xb\" (UniqueName: \"kubernetes.io/projected/c3b1f1cd-52ff-4efe-872f-7b767f340e58-kube-api-access-t72xb\") pod \"authorino-operator-7b7bb4bc77-vtm7p\" (UID: \"c3b1f1cd-52ff-4efe-872f-7b767f340e58\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:33.432793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.432753    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t72xb\" (UniqueName: \"kubernetes.io/projected/c3b1f1cd-52ff-4efe-872f-7b767f340e58-kube-api-access-t72xb\") pod \"authorino-operator-7b7bb4bc77-vtm7p\" (UID: \"c3b1f1cd-52ff-4efe-872f-7b767f340e58\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:33.445127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.445099    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t72xb\" (UniqueName: \"kubernetes.io/projected/c3b1f1cd-52ff-4efe-872f-7b767f340e58-kube-api-access-t72xb\") pod \"authorino-operator-7b7bb4bc77-vtm7p\" (UID: \"c3b1f1cd-52ff-4efe-872f-7b767f340e58\") " pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:33.601698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.601626    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg" event={"ID":"d74ad49a-0cce-4daa-a991-0e65c6c24765","Type":"ContainerStarted","Data":"e29c8dce34ac1762f4b5a7a43cfcaff36382cee90bb553497c43afabdab25a3b"}
Nov 01 22:26:33.602072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.601767    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:33.620427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.620397    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:33.636625 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.636580    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg" podStartSLOduration=2.525195867 podStartE2EDuration="7.636562271s" totalImagesPullingTime="4.475594475s" totalInitContainerRuntime="635.771929ms" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:26 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:33.636311977 +0000 UTC m=+250.378007597" watchObservedRunningTime="2026-11-01 22:26:33.636562271 +0000 UTC m=+250.378257893"
Nov 01 22:26:33.926754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:33.926714    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"]
Nov 01 22:26:33.930076 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:26:33.930035    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3b1f1cd_52ff_4efe_872f_7b767f340e58.slice/crio-e75d70050df6a5f8ca93bd6b066556280155f38ff4f9e3d184e3c8029fac5e12 WatchSource:0}: Error finding container e75d70050df6a5f8ca93bd6b066556280155f38ff4f9e3d184e3c8029fac5e12: Status 404 returned error can't find the container with id e75d70050df6a5f8ca93bd6b066556280155f38ff4f9e3d184e3c8029fac5e12
Nov 01 22:26:34.261607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:34.261567    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:34.391387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:34.391354    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:34.608252 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:34.608166    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p" event={"ID":"c3b1f1cd-52ff-4efe-872f-7b767f340e58","Type":"ContainerStarted","Data":"e75d70050df6a5f8ca93bd6b066556280155f38ff4f9e3d184e3c8029fac5e12"}
Nov 01 22:26:35.615950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:35.615914    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm" event={"ID":"86b517fe-edbf-4182-803d-f61b65a6eb09","Type":"ContainerStarted","Data":"00b3518cb6acd57f428b5a4fc55bc1318c02f90fdd0ef0fddbff693686d76a2d"}
Nov 01 22:26:35.635340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:35.635277    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-operator-674cd59dc5-jxxbm" podStartSLOduration=2.049104078 podStartE2EDuration="5.635262935s" totalImagesPullingTime="3.586158857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:35.633820441 +0000 UTC m=+252.375516062" watchObservedRunningTime="2026-11-01 22:26:35.635262935 +0000 UTC m=+252.376958554"
Nov 01 22:26:36.622914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:36.622873    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p" event={"ID":"c3b1f1cd-52ff-4efe-872f-7b767f340e58","Type":"ContainerStarted","Data":"f4a4d0c736c1408e2a8a100ebb214971870c89dc6d16008b1aa419925c04eab5"}
Nov 01 22:26:36.623311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:36.623171    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:36.641619 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:36.641578    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p" podStartSLOduration=1.203365763 podStartE2EDuration="3.641563658s" totalImagesPullingTime="2.438197895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:36.639079263 +0000 UTC m=+253.380774888" watchObservedRunningTime="2026-11-01 22:26:36.641563658 +0000 UTC m=+253.383259276"
Nov 01 22:26:37.271134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:37.271087    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:37.629610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:37.629524    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th" event={"ID":"f9f45aac-69b5-4da9-b8a2-7208b9236db8","Type":"ContainerStarted","Data":"1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704"}
Nov 01 22:26:37.630005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:37.629747    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:37.649036 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:37.648993    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th" podStartSLOduration=2.595847574 podStartE2EDuration="10.648978555s" totalImagesPullingTime="8.053130981s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:37.647839238 +0000 UTC m=+254.389534858" watchObservedRunningTime="2026-11-01 22:26:37.648978555 +0000 UTC m=+254.390674174"
Nov 01 22:26:38.824735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:38.824698    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:38.954600 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:38.954563    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:39.072407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:39.072373    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:39.923217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:39.923180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:40.012697 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:26:40.012663    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:26:40.041172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:40.041141    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:42.700907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:42.700869    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:44.230760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:44.230727    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:44.610745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:44.610673    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-8487c6b6d6-f98qg"
Nov 01 22:26:47.306299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:47.306267    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:47.428342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:47.428307    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:47.631675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:47.631602    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-7b7bb4bc77-vtm7p"
Nov 01 22:26:47.969639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:47.969604    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:48.100524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.100475    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:48.305279 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.305202    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:48.425548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.425508    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:48.636400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.636313    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:26:48.852187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.852155    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"]
Nov 01 22:26:48.929890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.929825    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"]
Nov 01 22:26:48.930029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:48.929930    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.075947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.075910    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hf8jz\" (UniqueName: \"kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz\") pod \"kuadrant-operator-controller-manager-678d87766f-dtjcv\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.076115 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.075965    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-dtjcv\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.176754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.176718    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hf8jz\" (UniqueName: \"kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz\") pod \"kuadrant-operator-controller-manager-678d87766f-dtjcv\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.176913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.176766    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-dtjcv\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.177148 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.177129    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-dtjcv\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.185804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.185749    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hf8jz\" (UniqueName: \"kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz\") pod \"kuadrant-operator-controller-manager-678d87766f-dtjcv\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.261500 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.261477    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:49.557624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.557572    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"]
Nov 01 22:26:49.560396 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:26:49.560364    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c6901d3_05eb_4d28_8543_76ec9d234180.slice/crio-c341cc9f6f5427a2f74673a7228f04395c51f621f55e333436f28e9a8a1fdf94 WatchSource:0}: Error finding container c341cc9f6f5427a2f74673a7228f04395c51f621f55e333436f28e9a8a1fdf94: Status 404 returned error can't find the container with id c341cc9f6f5427a2f74673a7228f04395c51f621f55e333436f28e9a8a1fdf94
Nov 01 22:26:49.686838 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.686798    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" event={"ID":"5c6901d3-05eb-4d28-8543-76ec9d234180","Type":"ContainerStarted","Data":"0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da"}
Nov 01 22:26:49.687024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.686861    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" event={"ID":"5c6901d3-05eb-4d28-8543-76ec9d234180","Type":"ContainerStarted","Data":"c341cc9f6f5427a2f74673a7228f04395c51f621f55e333436f28e9a8a1fdf94"}
Nov 01 22:26:49.687024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:49.687003    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:26:52.035906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.035872    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:52.163706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.163672    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:52.299735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.299659    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:52.421555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.421483    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" podStartSLOduration=4.421468589 podStartE2EDuration="4.421468589s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:49.708778599 +0000 UTC m=+266.450474219" watchObservedRunningTime="2026-11-01 22:26:52.421468589 +0000 UTC m=+269.163164208"
Nov 01 22:26:52.421754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.421719    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"]
Nov 01 22:26:52.444136 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.444103    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:52.459713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.459672    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"]
Nov 01 22:26:52.459869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.459784    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.464297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.464272    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4vdtg\""
Nov 01 22:26:52.575165 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.575084    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:52.602136 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602106    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602252 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602156    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602252 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602223    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602258    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602282    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602441 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602410    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602469    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602534    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5598\" (UniqueName: \"kubernetes.io/projected/a31555b0-ec20-4c38-a4e0-4235abb76d9a-kube-api-access-m5598\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.602589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.602571    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703259    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703309    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703336    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703375    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703393    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703418    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703462    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703510    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5598\" (UniqueName: \"kubernetes.io/projected/a31555b0-ec20-4c38-a4e0-4235abb76d9a-kube-api-access-m5598\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703546    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.703974 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.703940    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.704114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.704011    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-data\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.704114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.704049    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.704237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.704218    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.704297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.704246    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.705951 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.705922    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.707423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.707402    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.731518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.731467    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a31555b0-ec20-4c38-a4e0-4235abb76d9a-istio-token\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.731721 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.731697    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5598\" (UniqueName: \"kubernetes.io/projected/a31555b0-ec20-4c38-a4e0-4235abb76d9a-kube-api-access-m5598\") pod \"data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45\" (UID: \"a31555b0-ec20-4c38-a4e0-4235abb76d9a\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.759578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.759546    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:52.792540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.792502    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:52.992717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:52.992679    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:53.099065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.099014    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"]
Nov 01 22:26:53.110905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.110855    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:26:53.111044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.110946    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:26:53.111044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.110993    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:26:53.111811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.111784    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:53.376867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.376777    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:53.514370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.514338    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:53.578117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.578088    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:26:53.603048 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.603020    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:26:53.603198 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.603124    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.607222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.607198    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5wxtv\""
Nov 01 22:26:53.610791 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.610680    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Nov 01 22:26:53.612841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.612814    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:26:53.708021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.707994    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45" event={"ID":"a31555b0-ec20-4c38-a4e0-4235abb76d9a","Type":"ContainerStarted","Data":"7c1aef7adc812e3dd9813f57080abc7cfab3cbfd75b1989196cce5e22921394b"}
Nov 01 22:26:53.708190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.708044    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45" event={"ID":"a31555b0-ec20-4c38-a4e0-4235abb76d9a","Type":"ContainerStarted","Data":"a6d7df3c8fb4122f6165320da6abc92a66f3b22199a5eb97ab6c5c096acc9a73"}
Nov 01 22:26:53.715352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.715329    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjbrk\" (UniqueName: \"kubernetes.io/projected/1b60af0c-bab8-4187-a0c2-f64f8ae42fa3-kube-api-access-hjbrk\") pod \"limitador-limitador-7c9b87d574-spsmr\" (UID: \"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.715472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.715360    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b60af0c-bab8-4187-a0c2-f64f8ae42fa3-config-file\") pod \"limitador-limitador-7c9b87d574-spsmr\" (UID: \"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.728664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.728604    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45" podStartSLOduration=1.728591277 podStartE2EDuration="1.728591277s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:53.727341276 +0000 UTC m=+270.469036919" watchObservedRunningTime="2026-11-01 22:26:53.728591277 +0000 UTC m=+270.470286895"
Nov 01 22:26:53.793386 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.793352    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:53.816801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.816773    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjbrk\" (UniqueName: \"kubernetes.io/projected/1b60af0c-bab8-4187-a0c2-f64f8ae42fa3-kube-api-access-hjbrk\") pod \"limitador-limitador-7c9b87d574-spsmr\" (UID: \"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.816801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.816807    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b60af0c-bab8-4187-a0c2-f64f8ae42fa3-config-file\") pod \"limitador-limitador-7c9b87d574-spsmr\" (UID: \"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.817542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.817480    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b60af0c-bab8-4187-a0c2-f64f8ae42fa3-config-file\") pod \"limitador-limitador-7c9b87d574-spsmr\" (UID: \"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.829694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.829667    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjbrk\" (UniqueName: \"kubernetes.io/projected/1b60af0c-bab8-4187-a0c2-f64f8ae42fa3-kube-api-access-hjbrk\") pod \"limitador-limitador-7c9b87d574-spsmr\" (UID: \"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.930434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.930407    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:53.992321 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:53.992269    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:54.115224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:54.115191    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:54.202005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:54.201972    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:26:54.204039 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:26:54.204011    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b60af0c_bab8_4187_a0c2_f64f8ae42fa3.slice/crio-6ebc9a9c2495718d0355d780777ae261cb3688bb4cd014076a3ce1bee21bc154 WatchSource:0}: Error finding container 6ebc9a9c2495718d0355d780777ae261cb3688bb4cd014076a3ce1bee21bc154: Status 404 returned error can't find the container with id 6ebc9a9c2495718d0355d780777ae261cb3688bb4cd014076a3ce1bee21bc154
Nov 01 22:26:54.447907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:54.447823    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:54.714049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:54.714017    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr" event={"ID":"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3","Type":"ContainerStarted","Data":"6ebc9a9c2495718d0355d780777ae261cb3688bb4cd014076a3ce1bee21bc154"}
Nov 01 22:26:54.797780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:54.797750    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:55.238619 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:55.238583    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:55.722374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:55.722329    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr" event={"ID":"1b60af0c-bab8-4187-a0c2-f64f8ae42fa3","Type":"ContainerStarted","Data":"bdb845dae63d6ae641e373035a18fae05334b19432559421c3743af55f5630d5"}
Nov 01 22:26:55.722693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:55.722668    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:26:55.722784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:55.722715    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:55.723556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:55.723534    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45"
Nov 01 22:26:55.739047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:55.739005    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr" podStartSLOduration=1.5417726520000001 podStartE2EDuration="2.738992664s" totalImagesPullingTime="1.197220012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:26:55.737777612 +0000 UTC m=+272.479473228" watchObservedRunningTime="2026-11-01 22:26:55.738992664 +0000 UTC m=+272.480688282"
Nov 01 22:26:56.308536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:56.308477    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:56.426121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:56.426020    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:58.885391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:58.885359    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:26:59.003694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:26:59.003659    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:00.695335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:00.695306    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 22:27:00.744153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:00.744115    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"]
Nov 01 22:27:00.744423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:00.744393    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th" podUID="f9f45aac-69b5-4da9-b8a2-7208b9236db8" containerName="manager" containerID="cri-o://1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704" gracePeriod=10
Nov 01 22:27:00.991684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:00.991654    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:27:01.083354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.083322    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume\") pod \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") "
Nov 01 22:27:01.083578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.083404    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch\" (UniqueName: \"kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch\") pod \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\" (UID: \"f9f45aac-69b5-4da9-b8a2-7208b9236db8\") "
Nov 01 22:27:01.083855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.083829    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume" pod "f9f45aac-69b5-4da9-b8a2-7208b9236db8" (UID: "f9f45aac-69b5-4da9-b8a2-7208b9236db8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:27:01.085675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.085650    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch" pod "f9f45aac-69b5-4da9-b8a2-7208b9236db8" (UID: "f9f45aac-69b5-4da9-b8a2-7208b9236db8"). InnerVolumeSpecName "kube-api-access-44rch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:27:01.184424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.184386    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44rch\" (UniqueName: \"kubernetes.io/projected/f9f45aac-69b5-4da9-b8a2-7208b9236db8-kube-api-access-44rch\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:27:01.184424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.184420    5057 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9f45aac-69b5-4da9-b8a2-7208b9236db8-extensions-socket-volume\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:27:01.750389 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.750355    5057 generic.go:356] "Generic (PLEG): container finished" podID="f9f45aac-69b5-4da9-b8a2-7208b9236db8" containerID="1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704" exitCode=0
Nov 01 22:27:01.751016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.750403    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"
Nov 01 22:27:01.751016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.750433    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th" event={"ID":"f9f45aac-69b5-4da9-b8a2-7208b9236db8","Type":"ContainerDied","Data":"1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704"}
Nov 01 22:27:01.751016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.750487    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th" event={"ID":"f9f45aac-69b5-4da9-b8a2-7208b9236db8","Type":"ContainerDied","Data":"af5c250f24e803584e28bba83e1d3e11c08b79967df17bd8805fef88ec9a18e6"}
Nov 01 22:27:01.751016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.750530    5057 scope.go:122] "RemoveContainer" containerID="1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704"
Nov 01 22:27:01.759123 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.759104    5057 scope.go:122] "RemoveContainer" containerID="1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704"
Nov 01 22:27:01.759406 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:27:01.759385    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704\": container with ID starting with 1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704 not found: ID does not exist" containerID="1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704"
Nov 01 22:27:01.759451 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.759416    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704"} err="failed to get container status \"1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704\": rpc error: code = NotFound desc = could not find container \"1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704\": container with ID starting with 1bb14ff0215385d8aade6c0bc62075d61392c1d51f9ea800e3167a3325ca5704 not found: ID does not exist"
Nov 01 22:27:01.773105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.773082    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"]
Nov 01 22:27:01.783111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.780155    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-989th"]
Nov 01 22:27:01.791771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.791746    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:01.919944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:01.919908    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:03.395809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:03.395775    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9f45aac-69b5-4da9-b8a2-7208b9236db8" path="/var/lib/kubelet/pods/f9f45aac-69b5-4da9-b8a2-7208b9236db8/volumes"
Nov 01 22:27:06.438065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:06.438013    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:06.606283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:06.606239    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:06.727785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:06.727755    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-spsmr"
Nov 01 22:27:10.013258 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:27:10.013220    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:27:12.238078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:12.238046    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:15.261451 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:15.261414    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:15.373980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:15.373942    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:17.863870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:17.863832    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:17.995459 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:17.995406    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:21.059658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:21.059614    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:21.164169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:21.164135    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:23.402696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:23.402671    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:27:23.403608 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:23.403585    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:27:24.789071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:24.789030    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:24.907215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:24.907177    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:25.349410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.349369    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:25.406627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.406589    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"]
Nov 01 22:27:25.406981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.406968    5057 cpu_manager.go:414] "containerMap: removing container" podUID="f9f45aac-69b5-4da9-b8a2-7208b9236db8" containerName="manager"
Nov 01 22:27:25.407034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.406982    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9f45aac-69b5-4da9-b8a2-7208b9236db8" containerName="manager"
Nov 01 22:27:25.407072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.407051    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9f45aac-69b5-4da9-b8a2-7208b9236db8" containerName="manager"
Nov 01 22:27:25.411776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.411758    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.416044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.416016    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Nov 01 22:27:25.416404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.416390    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Nov 01 22:27:25.416684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.416662    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"]
Nov 01 22:27:25.490574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.490532    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/df79818f-3c33-4b20-8c29-929ae609823d-data\") pod \"postgres-7cdbf6dc44-455hj\" (UID: \"df79818f-3c33-4b20-8c29-929ae609823d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.490766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.490583    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8w67\" (UniqueName: \"kubernetes.io/projected/df79818f-3c33-4b20-8c29-929ae609823d-kube-api-access-k8w67\") pod \"postgres-7cdbf6dc44-455hj\" (UID: \"df79818f-3c33-4b20-8c29-929ae609823d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.512110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.512085    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Nov 01 22:27:25.512359 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.512342    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lclcx\""
Nov 01 22:27:25.519851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.519804    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:25.591818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.591776    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/df79818f-3c33-4b20-8c29-929ae609823d-data\") pod \"postgres-7cdbf6dc44-455hj\" (UID: \"df79818f-3c33-4b20-8c29-929ae609823d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.592004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.591838    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8w67\" (UniqueName: \"kubernetes.io/projected/df79818f-3c33-4b20-8c29-929ae609823d-kube-api-access-k8w67\") pod \"postgres-7cdbf6dc44-455hj\" (UID: \"df79818f-3c33-4b20-8c29-929ae609823d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.592211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.592189    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/df79818f-3c33-4b20-8c29-929ae609823d-data\") pod \"postgres-7cdbf6dc44-455hj\" (UID: \"df79818f-3c33-4b20-8c29-929ae609823d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.599776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.599712    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8w67\" (UniqueName: \"kubernetes.io/projected/df79818f-3c33-4b20-8c29-929ae609823d-kube-api-access-k8w67\") pod \"postgres-7cdbf6dc44-455hj\" (UID: \"df79818f-3c33-4b20-8c29-929ae609823d\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.725697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.725662    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:25.860372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.860289    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:25.973094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.973039    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:25.996425 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:25.996400    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"]
Nov 01 22:27:25.998765 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:27:25.998732    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf79818f_3c33_4b20_8c29_929ae609823d.slice/crio-d42d8ab21dfaa68eda562743ad6940d335126aa00b1372b187364d891c91f0ba WatchSource:0}: Error finding container d42d8ab21dfaa68eda562743ad6940d335126aa00b1372b187364d891c91f0ba: Status 404 returned error can't find the container with id d42d8ab21dfaa68eda562743ad6940d335126aa00b1372b187364d891c91f0ba
Nov 01 22:27:26.000300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:26.000276    5057 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 22:27:26.465398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:26.465361    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:26.578579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:26.578542    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:26.869629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:26.869547    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj" event={"ID":"df79818f-3c33-4b20-8c29-929ae609823d","Type":"ContainerStarted","Data":"d42d8ab21dfaa68eda562743ad6940d335126aa00b1372b187364d891c91f0ba"}
Nov 01 22:27:27.074943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:27.074897    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:27.190460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:27.190364    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:31.901423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:31.901386    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj" event={"ID":"df79818f-3c33-4b20-8c29-929ae609823d","Type":"ContainerStarted","Data":"6618f460cd751b6d621181bef6e06e55f58111d3d8099e760fd72c66986760ab"}
Nov 01 22:27:31.901839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:31.901455    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:31.921247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:31.921181    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj" podStartSLOduration=1.492178225 podStartE2EDuration="6.921162062s" totalImagesPullingTime="5.428983837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:27:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:27:31.919522025 +0000 UTC m=+308.661217645" watchObservedRunningTime="2026-11-01 22:27:31.921162062 +0000 UTC m=+308.662857682"
Nov 01 22:27:35.883807 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:35.883725    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:35.994754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:35.994707    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:36.466003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:36.465966    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:36.587019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:36.586976    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:37.934806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:37.934778    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-455hj"
Nov 01 22:27:40.013003 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:27:40.012961    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:27:44.969962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:44.969926    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:45.035209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.035175    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-579cf7c6d-fh79f"]
Nov 01 22:27:45.039003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.038980    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.051888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.051866    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Nov 01 22:27:45.052055 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.052036    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Nov 01 22:27:45.052880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.052867    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Nov 01 22:27:45.052929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.052873    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Nov 01 22:27:45.053135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.053120    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-m2mcq\""
Nov 01 22:27:45.053171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.053126    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Nov 01 22:27:45.088579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.088554    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-fh79f"]
Nov 01 22:27:45.179811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.179778    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:45.182809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.182791    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.182883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.182833    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.182986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.182966    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jttb\" (UniqueName: \"kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.284432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.284352    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.284611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.284441    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jttb\" (UniqueName: \"kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.284611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.284515    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.287031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.287001    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.287173 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.287042    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.293303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.293276    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jttb\" (UniqueName: \"kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb\") pod \"maas-controller-579cf7c6d-fh79f\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") " pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.297915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.297885    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:45.338921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.338883    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7b57d74f86-p96tx"]
Nov 01 22:27:45.342433 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.342417    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.351335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.351302    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:45.354467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.354444    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b57d74f86-p96tx"]
Nov 01 22:27:45.486389 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.486347    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.486576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.486403    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrl4x\" (UniqueName: \"kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.486576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.486465    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.587762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.587684    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.587762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.587725    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrl4x\" (UniqueName: \"kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.587762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.587747    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.590284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.590251    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.590284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.590269    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.604481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.604456    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrl4x\" (UniqueName: \"kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x\") pod \"maas-controller-7b57d74f86-p96tx\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") " pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.647935 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.647907    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-fh79f"]
Nov 01 22:27:45.650697 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:27:45.650661    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb12e2bb0_0a79_4f62_94d1_e1c0775bd791.slice/crio-efcc75d03eec1911193d69213d98328bb7b6f45fce06ff8f46c27b0e7cbe336a WatchSource:0}: Error finding container efcc75d03eec1911193d69213d98328bb7b6f45fce06ff8f46c27b0e7cbe336a: Status 404 returned error can't find the container with id efcc75d03eec1911193d69213d98328bb7b6f45fce06ff8f46c27b0e7cbe336a
Nov 01 22:27:45.656964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.656939    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:45.965550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.965487    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-fh79f" event={"ID":"b12e2bb0-0a79-4f62-94d1-e1c0775bd791","Type":"ContainerStarted","Data":"efcc75d03eec1911193d69213d98328bb7b6f45fce06ff8f46c27b0e7cbe336a"}
Nov 01 22:27:45.969321 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:45.969296    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7b57d74f86-p96tx"]
Nov 01 22:27:45.971655 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:27:45.971628    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda741756c_9713_4bad_981f_0f1b25ffa080.slice/crio-ef55059d421e095467179f12fb7474da11c09bb50a3aa192be6acf658f4c86df WatchSource:0}: Error finding container ef55059d421e095467179f12fb7474da11c09bb50a3aa192be6acf658f4c86df: Status 404 returned error can't find the container with id ef55059d421e095467179f12fb7474da11c09bb50a3aa192be6acf658f4c86df
Nov 01 22:27:46.977639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:46.977598    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b57d74f86-p96tx" event={"ID":"a741756c-9713-4bad-981f-0f1b25ffa080","Type":"ContainerStarted","Data":"ef55059d421e095467179f12fb7474da11c09bb50a3aa192be6acf658f4c86df"}
Nov 01 22:27:48.988989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:48.988940    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b57d74f86-p96tx" event={"ID":"a741756c-9713-4bad-981f-0f1b25ffa080","Type":"ContainerStarted","Data":"b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0"}
Nov 01 22:27:48.989392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:48.989038    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:48.990416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:48.990383    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-fh79f" event={"ID":"b12e2bb0-0a79-4f62-94d1-e1c0775bd791","Type":"ContainerStarted","Data":"3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815"}
Nov 01 22:27:48.990590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:48.990520    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:27:49.056572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.056518    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-7b57d74f86-p96tx" podStartSLOduration=1.23614468 podStartE2EDuration="4.056485685s" totalImagesPullingTime="2.820341005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:27:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:27:49.055161019 +0000 UTC m=+325.796856639" watchObservedRunningTime="2026-11-01 22:27:49.056485685 +0000 UTC m=+325.798181345"
Nov 01 22:27:49.062348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.062307    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:49.106001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.105945    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-579cf7c6d-fh79f" podStartSLOduration=2.088879581 podStartE2EDuration="5.105930185s" totalImagesPullingTime="3.017050604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:27:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:27:49.103476527 +0000 UTC m=+325.845172149" watchObservedRunningTime="2026-11-01 22:27:49.105930185 +0000 UTC m=+325.847625804"
Nov 01 22:27:49.240455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.240360    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:49.706048 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.705907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:49.777170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.777137    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"]
Nov 01 22:27:49.782269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.782244    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:49.806762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.806734    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"]
Nov 01 22:27:49.836602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.836572    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Nov 01 22:27:49.837694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.837666    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d9phd\""
Nov 01 22:27:49.942472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.942437    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:49.943481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.943454    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-w94m5\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:49.943664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:49.943637    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xps99\" (UniqueName: \"kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99\") pod \"llmisvc-controller-manager-869d5dcc5d-w94m5\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:50.044673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.044585    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xps99\" (UniqueName: \"kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99\") pod \"llmisvc-controller-manager-869d5dcc5d-w94m5\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:50.044673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.044661    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-w94m5\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:50.047146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.047120    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-w94m5\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:50.059373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.059349    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xps99\" (UniqueName: \"kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99\") pod \"llmisvc-controller-manager-869d5dcc5d-w94m5\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:50.112339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.112303    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:50.222426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.222392    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:50.368870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.368773    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:50.387560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:50.387532    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"]
Nov 01 22:27:50.389620 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:27:50.389591    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod01b46f3a_3e06_4a64_a54c_83e3ee77eb10.slice/crio-189d72132ff54226e1de0f9f2709bec227b0e9018fdc2f6f34e02824da9053cd WatchSource:0}: Error finding container 189d72132ff54226e1de0f9f2709bec227b0e9018fdc2f6f34e02824da9053cd: Status 404 returned error can't find the container with id 189d72132ff54226e1de0f9f2709bec227b0e9018fdc2f6f34e02824da9053cd
Nov 01 22:27:51.003359 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:51.003321    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5" event={"ID":"01b46f3a-3e06-4a64-a54c-83e3ee77eb10","Type":"ContainerStarted","Data":"189d72132ff54226e1de0f9f2709bec227b0e9018fdc2f6f34e02824da9053cd"}
Nov 01 22:27:51.578385 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:51.578339    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:51.705368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:51.705330    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:51.827676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:51.827635    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:53.138921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.138861    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:53.256551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.256515    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:53.517794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.517707    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"]
Nov 01 22:27:53.522245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.522218    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.532976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.532913    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Nov 01 22:27:53.532976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.532977    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Nov 01 22:27:53.533187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.533093    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-nrgkp\""
Nov 01 22:27:53.536387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.534280    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Nov 01 22:27:53.536387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.535735    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"]
Nov 01 22:27:53.601525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.601482    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:53.680533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.680482    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0983b1d5-dfad-454e-b71a-be502461bd6b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.680533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.680534    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9gdx\" (UniqueName: \"kubernetes.io/projected/0983b1d5-dfad-454e-b71a-be502461bd6b-kube-api-access-g9gdx\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.680734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.680640    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/0983b1d5-dfad-454e-b71a-be502461bd6b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.680734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.680675    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0983b1d5-dfad-454e-b71a-be502461bd6b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.781723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.781640    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0983b1d5-dfad-454e-b71a-be502461bd6b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.781723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.781677    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9gdx\" (UniqueName: \"kubernetes.io/projected/0983b1d5-dfad-454e-b71a-be502461bd6b-kube-api-access-g9gdx\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.781723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.781703    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/0983b1d5-dfad-454e-b71a-be502461bd6b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.782001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.781818    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0983b1d5-dfad-454e-b71a-be502461bd6b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.782592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.782557    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0983b1d5-dfad-454e-b71a-be502461bd6b-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.782795 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.782774    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/0983b1d5-dfad-454e-b71a-be502461bd6b-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.784354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.784329    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0983b1d5-dfad-454e-b71a-be502461bd6b-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.789546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.789526    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9gdx\" (UniqueName: \"kubernetes.io/projected/0983b1d5-dfad-454e-b71a-be502461bd6b-kube-api-access-g9gdx\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-lj699\" (UID: \"0983b1d5-dfad-454e-b71a-be502461bd6b\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:53.832414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.832366    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:27:53.850331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:53.850300    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:27:54.019645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:54.019610    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5" event={"ID":"01b46f3a-3e06-4a64-a54c-83e3ee77eb10","Type":"ContainerStarted","Data":"c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625"}
Nov 01 22:27:54.019800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:54.019712    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:27:54.036571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:54.036458    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5" podStartSLOduration=2.486553756 podStartE2EDuration="5.036443597s" totalImagesPullingTime="2.549889841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:27:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:27:54.03507602 +0000 UTC m=+330.776771651" watchObservedRunningTime="2026-11-01 22:27:54.036443597 +0000 UTC m=+330.778139215"
Nov 01 22:27:54.122957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:54.122930    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"]
Nov 01 22:27:54.125190 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:27:54.125157    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0983b1d5_dfad_454e_b71a_be502461bd6b.slice/crio-124580ce7b83b6ab3231c33b2468e6ffdcb5f57fb6dd55dfa0260999ab912ddf WatchSource:0}: Error finding container 124580ce7b83b6ab3231c33b2468e6ffdcb5f57fb6dd55dfa0260999ab912ddf: Status 404 returned error can't find the container with id 124580ce7b83b6ab3231c33b2468e6ffdcb5f57fb6dd55dfa0260999ab912ddf
Nov 01 22:27:55.028434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:55.028376    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699" event={"ID":"0983b1d5-dfad-454e-b71a-be502461bd6b","Type":"ContainerStarted","Data":"124580ce7b83b6ab3231c33b2468e6ffdcb5f57fb6dd55dfa0260999ab912ddf"}
Nov 01 22:27:59.998508 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:59.998451    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:27:59.999616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:27:59.999590    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:28:00.062955 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.062924    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-fh79f"]
Nov 01 22:28:00.063390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.063135    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-579cf7c6d-fh79f" podUID="b12e2bb0-0a79-4f62-94d1-e1c0775bd791" containerName="manager" containerID="cri-o://3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815" gracePeriod=30
Nov 01 22:28:00.341543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.341514    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:28:00.445531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.445294    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb\" (UniqueName: \"kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb\") pod \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") "
Nov 01 22:28:00.445531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.445466    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs\") pod \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") "
Nov 01 22:28:00.445531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.445521    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert\") pod \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\" (UID: \"b12e2bb0-0a79-4f62-94d1-e1c0775bd791\") "
Nov 01 22:28:00.448749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.448706    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs" pod "b12e2bb0-0a79-4f62-94d1-e1c0775bd791" (UID: "b12e2bb0-0a79-4f62-94d1-e1c0775bd791"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:28:00.450032 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.450000    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert" pod "b12e2bb0-0a79-4f62-94d1-e1c0775bd791" (UID: "b12e2bb0-0a79-4f62-94d1-e1c0775bd791"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:28:00.450215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.450191    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb" pod "b12e2bb0-0a79-4f62-94d1-e1c0775bd791" (UID: "b12e2bb0-0a79-4f62-94d1-e1c0775bd791"). InnerVolumeSpecName "kube-api-access-6jttb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:28:00.546938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.546891    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:28:00.546938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.546938    5057 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-webhook-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:28:00.547192 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:00.546957    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jttb\" (UniqueName: \"kubernetes.io/projected/b12e2bb0-0a79-4f62-94d1-e1c0775bd791-kube-api-access-6jttb\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:28:01.069812 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.069770    5057 generic.go:356] "Generic (PLEG): container finished" podID="b12e2bb0-0a79-4f62-94d1-e1c0775bd791" containerID="3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815" exitCode=0
Nov 01 22:28:01.070231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.069820    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-579cf7c6d-fh79f"
Nov 01 22:28:01.070231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.069847    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-fh79f" event={"ID":"b12e2bb0-0a79-4f62-94d1-e1c0775bd791","Type":"ContainerDied","Data":"3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815"}
Nov 01 22:28:01.070231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.069906    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-579cf7c6d-fh79f" event={"ID":"b12e2bb0-0a79-4f62-94d1-e1c0775bd791","Type":"ContainerDied","Data":"efcc75d03eec1911193d69213d98328bb7b6f45fce06ff8f46c27b0e7cbe336a"}
Nov 01 22:28:01.070231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.069940    5057 scope.go:122] "RemoveContainer" containerID="3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815"
Nov 01 22:28:01.080118 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.080097    5057 scope.go:122] "RemoveContainer" containerID="3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815"
Nov 01 22:28:01.080423 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:28:01.080404    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815\": container with ID starting with 3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815 not found: ID does not exist" containerID="3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815"
Nov 01 22:28:01.080509 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.080435    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815"} err="failed to get container status \"3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815\": rpc error: code = NotFound desc = could not find container \"3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815\": container with ID starting with 3045030e9d77721e0f862a96ffc1706acb73d39bea947ff31c9c70995ebcc815 not found: ID does not exist"
Nov 01 22:28:01.091800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.091772    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-fh79f"]
Nov 01 22:28:01.094025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.093999    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-579cf7c6d-fh79f"]
Nov 01 22:28:01.396311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:01.396238    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b12e2bb0-0a79-4f62-94d1-e1c0775bd791" path="/var/lib/kubelet/pods/b12e2bb0-0a79-4f62-94d1-e1c0775bd791/volumes"
Nov 01 22:28:07.108798 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:07.108761    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699" event={"ID":"0983b1d5-dfad-454e-b71a-be502461bd6b","Type":"ContainerStarted","Data":"d44857803d133384ed4cd539c6d32b2e7020d70d99881a2e75a3f1c5de5203c4"}
Nov 01 22:28:07.109176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:07.108906    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:28:07.126172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:07.126128    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699" podStartSLOduration=1.22450624 podStartE2EDuration="14.126115054s" totalImagesPullingTime="12.901608814s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:28:07.124894312 +0000 UTC m=+343.866589929" watchObservedRunningTime="2026-11-01 22:28:07.126115054 +0000 UTC m=+343.867810667"
Nov 01 22:28:10.013775 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:28:10.013731    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:28:12.273740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:12.273676    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:18.118754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:18.118715    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-lj699"
Nov 01 22:28:21.052947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:21.052905    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:21.182394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:21.182354    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:22.328117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:22.328086    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:22.460561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:22.460524    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:22.585777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:22.585692    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:23.226614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:23.226576    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:28:25.030765 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:25.030736    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:28:33.205099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.205064    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:33.333564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.333529    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:28:33.637553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.637432    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"]
Nov 01 22:28:33.637858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.637845    5057 cpu_manager.go:414] "containerMap: removing container" podUID="b12e2bb0-0a79-4f62-94d1-e1c0775bd791" containerName="manager"
Nov 01 22:28:33.637901 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.637860    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b12e2bb0-0a79-4f62-94d1-e1c0775bd791" containerName="manager"
Nov 01 22:28:33.637942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.637934    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="b12e2bb0-0a79-4f62-94d1-e1c0775bd791" containerName="manager"
Nov 01 22:28:33.641556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.641530    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.650025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.650000    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"]
Nov 01 22:28:33.662162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.662140    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Nov 01 22:28:33.663134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.663115    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Nov 01 22:28:33.663204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.663128    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q7x7m\""
Nov 01 22:28:33.745440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.745409    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.745626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.745446    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlvnl\" (UniqueName: \"kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.745626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.745471    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.745710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.745666    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.846909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.846870    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.846909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.846914    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlvnl\" (UniqueName: \"kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.847122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.846948    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.847122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.847049    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.847734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.847712    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.850325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.850291    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.857944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.857915    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlvnl\" (UniqueName: \"kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.858068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.858047    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls\") pod \"maas-api-54c8494dc4-c6skz\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") " pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:33.956132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:33.956099    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:34.230116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:34.230034    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"]
Nov 01 22:28:34.232619 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:28:34.232586    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd57b4deb_05d4_489f_add3_e43743fc7ece.slice/crio-d4c10ec85d10a2429eb9b47af3fc88d0e14f7da759046c326159c7fe3bbf9cb8 WatchSource:0}: Error finding container d4c10ec85d10a2429eb9b47af3fc88d0e14f7da759046c326159c7fe3bbf9cb8: Status 404 returned error can't find the container with id d4c10ec85d10a2429eb9b47af3fc88d0e14f7da759046c326159c7fe3bbf9cb8
Nov 01 22:28:34.238454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:34.238429    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" event={"ID":"d57b4deb-05d4-489f-add3-e43743fc7ece","Type":"ContainerStarted","Data":"d4c10ec85d10a2429eb9b47af3fc88d0e14f7da759046c326159c7fe3bbf9cb8"}
Nov 01 22:28:36.251024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:36.250984    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" event={"ID":"d57b4deb-05d4-489f-add3-e43743fc7ece","Type":"ContainerStarted","Data":"7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1"}
Nov 01 22:28:36.251521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:36.251102    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:40.014896 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:28:40.014851    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:28:42.261757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:42.261725    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 22:28:42.299699 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:42.299646    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" podStartSLOduration=7.836461275 podStartE2EDuration="9.299634093s" totalImagesPullingTime="1.463172818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:28:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:28:36.266460746 +0000 UTC m=+373.008156366" watchObservedRunningTime="2026-11-01 22:28:42.299634093 +0000 UTC m=+379.041329711"
Nov 01 22:28:57.213782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.213743    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"]
Nov 01 22:28:57.245132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.245098    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"]
Nov 01 22:28:57.245309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.245237    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.342806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.342776    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bca932e-0124-45e7-8879-2df04b4706da-cert\") pod \"llmisvc-controller-manager-6466c554d8-sc9w7\" (UID: \"9bca932e-0124-45e7-8879-2df04b4706da\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.343008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.342836    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5xlq\" (UniqueName: \"kubernetes.io/projected/9bca932e-0124-45e7-8879-2df04b4706da-kube-api-access-r5xlq\") pod \"llmisvc-controller-manager-6466c554d8-sc9w7\" (UID: \"9bca932e-0124-45e7-8879-2df04b4706da\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.443312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.443279    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bca932e-0124-45e7-8879-2df04b4706da-cert\") pod \"llmisvc-controller-manager-6466c554d8-sc9w7\" (UID: \"9bca932e-0124-45e7-8879-2df04b4706da\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.443542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.443377    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5xlq\" (UniqueName: \"kubernetes.io/projected/9bca932e-0124-45e7-8879-2df04b4706da-kube-api-access-r5xlq\") pod \"llmisvc-controller-manager-6466c554d8-sc9w7\" (UID: \"9bca932e-0124-45e7-8879-2df04b4706da\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.445995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.445971    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9bca932e-0124-45e7-8879-2df04b4706da-cert\") pod \"llmisvc-controller-manager-6466c554d8-sc9w7\" (UID: \"9bca932e-0124-45e7-8879-2df04b4706da\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.452090 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.452064    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5xlq\" (UniqueName: \"kubernetes.io/projected/9bca932e-0124-45e7-8879-2df04b4706da-kube-api-access-r5xlq\") pod \"llmisvc-controller-manager-6466c554d8-sc9w7\" (UID: \"9bca932e-0124-45e7-8879-2df04b4706da\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.558785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.558690    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:57.837171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:57.837142    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"]
Nov 01 22:28:57.839588 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:28:57.839556    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bca932e_0124_45e7_8879_2df04b4706da.slice/crio-1288915a6dcbaba1b9b99097b5b8f3826e7a072cd8f063df37f5db0608e99248 WatchSource:0}: Error finding container 1288915a6dcbaba1b9b99097b5b8f3826e7a072cd8f063df37f5db0608e99248: Status 404 returned error can't find the container with id 1288915a6dcbaba1b9b99097b5b8f3826e7a072cd8f063df37f5db0608e99248
Nov 01 22:28:58.353149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:58.353114    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7" event={"ID":"9bca932e-0124-45e7-8879-2df04b4706da","Type":"ContainerStarted","Data":"34992a3bc48f335c62cc390ce067c49ee5d47042a8e5d2eee7e9b3839f2a53ca"}
Nov 01 22:28:58.353656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:58.353169    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7" event={"ID":"9bca932e-0124-45e7-8879-2df04b4706da","Type":"ContainerStarted","Data":"1288915a6dcbaba1b9b99097b5b8f3826e7a072cd8f063df37f5db0608e99248"}
Nov 01 22:28:58.353656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:58.353320    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:28:58.370772 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:28:58.370720    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7" podStartSLOduration=1.023263468 podStartE2EDuration="1.370702663s" totalImagesPullingTime="347.439195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:28:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:28:58.370299917 +0000 UTC m=+395.111995536" watchObservedRunningTime="2026-11-01 22:28:58.370702663 +0000 UTC m=+395.112398279"
Nov 01 22:29:00.485736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:00.485687    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:00.656533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:00.656481    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:04.985242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:04.985190    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:05.149045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:05.149009    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:10.012978 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:29:10.012942    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:29:12.369864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:12.369824    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:29.360471 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.360437    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-sc9w7"
Nov 01 22:29:29.406174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.406138    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"]
Nov 01 22:29:29.406675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.406365    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5" podUID="01b46f3a-3e06-4a64-a54c-83e3ee77eb10" containerName="manager" containerID="cri-o://c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625" gracePeriod=30
Nov 01 22:29:29.661913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.661887    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:29:29.733539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.733477    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert\" (UniqueName: \"kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert\") pod \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") "
Nov 01 22:29:29.733714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.733579    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99\" (UniqueName: \"kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99\") pod \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\" (UID: \"01b46f3a-3e06-4a64-a54c-83e3ee77eb10\") "
Nov 01 22:29:29.735618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.735589    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert" pod "01b46f3a-3e06-4a64-a54c-83e3ee77eb10" (UID: "01b46f3a-3e06-4a64-a54c-83e3ee77eb10"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:29:29.735746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.735636    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99" pod "01b46f3a-3e06-4a64-a54c-83e3ee77eb10" (UID: "01b46f3a-3e06-4a64-a54c-83e3ee77eb10"). InnerVolumeSpecName "kube-api-access-xps99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:29:29.834834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.834791    5057 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:29:29.834834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:29.834823    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xps99\" (UniqueName: \"kubernetes.io/projected/01b46f3a-3e06-4a64-a54c-83e3ee77eb10-kube-api-access-xps99\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:29:30.241195 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.241141    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:30.490065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.490015    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:30.506977 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.506865    5057 generic.go:356] "Generic (PLEG): container finished" podID="01b46f3a-3e06-4a64-a54c-83e3ee77eb10" containerID="c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625" exitCode=0
Nov 01 22:29:30.507121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.507049    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5" event={"ID":"01b46f3a-3e06-4a64-a54c-83e3ee77eb10","Type":"ContainerDied","Data":"c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625"}
Nov 01 22:29:30.507190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.507140    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5" event={"ID":"01b46f3a-3e06-4a64-a54c-83e3ee77eb10","Type":"ContainerDied","Data":"189d72132ff54226e1de0f9f2709bec227b0e9018fdc2f6f34e02824da9053cd"}
Nov 01 22:29:30.507190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.507175    5057 scope.go:122] "RemoveContainer" containerID="c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625"
Nov 01 22:29:30.509701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.507369    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"
Nov 01 22:29:30.517285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.517263    5057 scope.go:122] "RemoveContainer" containerID="c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625"
Nov 01 22:29:30.517569 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:29:30.517547    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625\": container with ID starting with c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625 not found: ID does not exist" containerID="c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625"
Nov 01 22:29:30.517637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.517580    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625"} err="failed to get container status \"c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625\": rpc error: code = NotFound desc = could not find container \"c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625\": container with ID starting with c2e5d7aac24840c11a8710f37f11f07a4dde0cfec6690787dfd204003e85c625 not found: ID does not exist"
Nov 01 22:29:30.530489 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.530461    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"]
Nov 01 22:29:30.535045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:30.535016    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-w94m5"]
Nov 01 22:29:31.395344 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:31.395313    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01b46f3a-3e06-4a64-a54c-83e3ee77eb10" path="/var/lib/kubelet/pods/01b46f3a-3e06-4a64-a54c-83e3ee77eb10/volumes"
Nov 01 22:29:40.013444 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:29:40.013405    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:29:52.034135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:52.034098    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:52.381751 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:52.381666    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:29:52.895141 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:52.895102    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:54.481631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:54.481596    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:54.922847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:54.922742    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:29:55.795101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:55.795059    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:56.340077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:56.340045    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:56.870144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:56.870102    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:57.987348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:57.987317    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:29:59.115781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:59.115735    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:29:59.307254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:29:59.307210    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:02.705013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:02.704974    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:02.835822 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:02.835784    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:03.733741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:03.733706    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:30:10.012790 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:30:10.012752    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:30:12.377711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:12.377672    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:13.224809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:13.224768    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:13.348256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:13.348212    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:19.637842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:19.637802    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:19.783367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:19.783318    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:22.499444 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.499409    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"]
Nov 01 22:30:22.499878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.499849    5057 cpu_manager.go:414] "containerMap: removing container" podUID="01b46f3a-3e06-4a64-a54c-83e3ee77eb10" containerName="manager"
Nov 01 22:30:22.499878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.499863    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01b46f3a-3e06-4a64-a54c-83e3ee77eb10" containerName="manager"
Nov 01 22:30:22.499975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.499963    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="01b46f3a-3e06-4a64-a54c-83e3ee77eb10" containerName="manager"
Nov 01 22:30:22.503294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.503272    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.512502 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.512468    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-flhqn\""
Nov 01 22:30:22.523127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.522309    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"]
Nov 01 22:30:22.534547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534518    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lkh\" (UniqueName: \"kubernetes.io/projected/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-kube-api-access-25lkh\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.534703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534553    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.534703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534576    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.534826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534701    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.534826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534723    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.534826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534747    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.534826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534811    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.535110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534859    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.535110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.534931    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.635853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635815    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lkh\" (UniqueName: \"kubernetes.io/projected/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-kube-api-access-25lkh\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.635853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635856    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635878    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635916    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635938    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635961    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.635988    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636035    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636064    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636432    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636455    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636580    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636628    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.636925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.636900    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.638258 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.638236    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.638580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.638560    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.647892 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.647866    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.648219 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.648196    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lkh\" (UniqueName: \"kubernetes.io/projected/a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec-kube-api-access-25lkh\") pod \"maas-default-gateway-openshift-default-58847f9fc5-d6k7r\" (UID: \"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:22.698029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.697990    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:22.817227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:22.817136    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:23.094594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.094512    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"]
Nov 01 22:30:23.098779 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:30:23.098749    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f080e5_a870_4c58_9f4e_2ba21bf7a0ec.slice/crio-e6bfa24e77a3574f9cc2137f81d4777e84278867ca2eba82df8e2e801a2fbdbb WatchSource:0}: Error finding container e6bfa24e77a3574f9cc2137f81d4777e84278867ca2eba82df8e2e801a2fbdbb: Status 404 returned error can't find the container with id e6bfa24e77a3574f9cc2137f81d4777e84278867ca2eba82df8e2e801a2fbdbb
Nov 01 22:30:23.101075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.101039    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:30:23.101178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.101127    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:30:23.101244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.101174    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:30:23.777732 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.777694    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" event={"ID":"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec","Type":"ContainerStarted","Data":"5880690b8d5ba30f15ce8af12efe786669ff8a02089b24018f123687987316d2"}
Nov 01 22:30:23.778273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.777745    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" event={"ID":"a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec","Type":"ContainerStarted","Data":"e6bfa24e77a3574f9cc2137f81d4777e84278867ca2eba82df8e2e801a2fbdbb"}
Nov 01 22:30:23.797439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.797389    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podStartSLOduration=1.797374341 podStartE2EDuration="1.797374341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:30:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:30:23.797081299 +0000 UTC m=+480.538776916" watchObservedRunningTime="2026-11-01 22:30:23.797374341 +0000 UTC m=+480.539069959"
Nov 01 22:30:23.842607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:23.842573    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:24.506979 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:24.506943    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:24.841577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:24.840677    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 01 22:30:24.841577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:24.840755    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 01 22:30:25.839945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:25.839901    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 01 22:30:25.840275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:25.840236    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 01 22:30:26.828971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:26.828928    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 01 22:30:26.829620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:26.829584    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 01 22:30:27.840518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:27.828878    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 01 22:30:27.840518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:27.828943    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 01 22:30:28.833979 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:28.833936    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 01 22:30:28.834267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:28.834237    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 01 22:30:29.817809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:29.817771    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Nov 01 22:30:29.818203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:29.817831    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Nov 01 22:30:30.134510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:30.134411    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:31.440670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:31.440636    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:31.441116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:31.441065    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:31.442518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:31.442478    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r"
Nov 01 22:30:37.517369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.517275    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"]
Nov 01 22:30:37.521319 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.521288    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.533533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.533506    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"]
Nov 01 22:30:37.566609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566573    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.566806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566614    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.566806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566715    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.566806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566742    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.566806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566766    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.566982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566885    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9sb4\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.566982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.566965    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.567071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.567018    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.567132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.567076    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.667862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.667818    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.667862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.667861    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.667895    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9sb4\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.667957    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668016    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668041    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668097    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668122    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668174    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668340    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668420    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668632    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668852 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668828    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.668971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.668951    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.670707 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.670680    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.670794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.670752    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.694264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.694232    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.694398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.694369    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9sb4\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4\") pod \"maas-default-gateway-openshift-default-58847f9fc5-tpppl\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:37.836992 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:37.836890    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:38.115140 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.115110    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"]
Nov 01 22:30:38.116960 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:30:38.116932    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a10a58_ea8f_437c_bb3e_cd6347d6bf69.slice/crio-c84b4fd0a20e60157f5bf591e03eed9fcb3198a503e3a442355f70015b3b04d4 WatchSource:0}: Error finding container c84b4fd0a20e60157f5bf591e03eed9fcb3198a503e3a442355f70015b3b04d4: Status 404 returned error can't find the container with id c84b4fd0a20e60157f5bf591e03eed9fcb3198a503e3a442355f70015b3b04d4
Nov 01 22:30:38.119281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.119246    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:30:38.119362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.119315    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:30:38.119362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.119347    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:30:38.850622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.850579    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" event={"ID":"75a10a58-ea8f-437c-bb3e-cd6347d6bf69","Type":"ContainerStarted","Data":"95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6"}
Nov 01 22:30:38.851036 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.850642    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" event={"ID":"75a10a58-ea8f-437c-bb3e-cd6347d6bf69","Type":"ContainerStarted","Data":"c84b4fd0a20e60157f5bf591e03eed9fcb3198a503e3a442355f70015b3b04d4"}
Nov 01 22:30:38.873067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:38.873014    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podStartSLOduration=1.8730008599999999 podStartE2EDuration="1.87300086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:30:38.871839653 +0000 UTC m=+495.613535274" watchObservedRunningTime="2026-11-01 22:30:38.87300086 +0000 UTC m=+495.614696479"
Nov 01 22:30:39.839516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:39.837921    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:39.842658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:39.842619    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 01 22:30:39.842796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:39.842686    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 01 22:30:40.014990 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:30:40.014940    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:30:40.842662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:40.842618    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 01 22:30:40.842819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:40.842692    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 01 22:30:41.842688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:41.842646    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 01 22:30:41.843112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:41.842721    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 01 22:30:42.843710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:42.843670    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 01 22:30:42.844353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:42.844311    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 01 22:30:43.842914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:43.842867    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 01 22:30:43.843146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:43.842934    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 01 22:30:44.494557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:44.494521    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:44.838254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:44.838162    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Nov 01 22:30:44.838254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:44.838219    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Nov 01 22:30:44.907180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:44.907133    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:46.475044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:46.475011    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:46.475532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:46.475368    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:46.476373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:46.476349    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 22:30:46.603809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:46.603759    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:46.788977 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:46.788895    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:48.040576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:48.040540    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:30:48.268172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:30:48.268132    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:10.013948 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:31:10.013912    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:31:12.433019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:12.432988    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:13.198773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:13.198741    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:13.307327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:13.307291    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:13.673746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:13.673716    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:13.800434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:13.800391    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:13.938378 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:13.938266    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:14.100134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:14.100098    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:14.233769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:14.233722    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:14.468878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:14.468845    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:14.645642 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:14.645554    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:16.880798 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:16.880760    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:17.009998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:17.009960    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:17.461928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:17.461893    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:17.675792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:17.675753    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:17.917776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:17.917682    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:18.189108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:18.189024    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:18.318750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:18.318717    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:18.476057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:18.476007    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:18.631659 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:18.631611    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:18.764537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:18.764422    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:19.102619 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:19.102520    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:19.245330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:19.245284    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:19.703251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:19.703208    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:19.836377 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:19.836336    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:20.188129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:20.188037    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:20.326721 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:20.326681    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:20.876392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:20.876358    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.034837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.034762    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.053445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.053404    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"]
Nov 01 22:31:21.058457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.058413    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"]
Nov 01 22:31:21.058669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.058615    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.093345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.093289    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-aa441b-serving-cert\""
Nov 01 22:31:21.161107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.161008    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.161271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.161176    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.161271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.161211    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bklc5\" (UniqueName: \"kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.161365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.161311    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.163176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.163144    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.262436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.262376    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.262888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.262449    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.262888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.262556    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.262888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.262577    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bklc5\" (UniqueName: \"kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.263271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.263248    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.265129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.265106    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.265226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.265163    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.271472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.271445    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bklc5\" (UniqueName: \"kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5\") pod \"maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.285352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.285310    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.374360 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.374325    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:21.500835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.499376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.636731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.636698    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.670945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.670917    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"]
Nov 01 22:31:21.673601 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:31:21.673567    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ef56ea1_0f1e_4553_8f34_b2a508dfcff8.slice/crio-47d5960a44dcde6e3def487c141a40ae3376f07e6bb2395fa3fb06652c946a39 WatchSource:0}: Error finding container 47d5960a44dcde6e3def487c141a40ae3376f07e6bb2395fa3fb06652c946a39: Status 404 returned error can't find the container with id 47d5960a44dcde6e3def487c141a40ae3376f07e6bb2395fa3fb06652c946a39
Nov 01 22:31:21.851156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.851114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:21.983247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:21.983200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:22.091938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:22.091909    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg" event={"ID":"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8","Type":"ContainerStarted","Data":"47d5960a44dcde6e3def487c141a40ae3376f07e6bb2395fa3fb06652c946a39"}
Nov 01 22:31:22.095969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:22.095939    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:22.190768 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:22.190733    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:31:22.294527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:22.294459    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:22.885206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:22.885149    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:23.098565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.098531    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg" event={"ID":"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8","Type":"ContainerStarted","Data":"c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db"}
Nov 01 22:31:23.098749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.098709    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:23.120467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.120404    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg" podStartSLOduration=1.7109599530000001 podStartE2EDuration="2.120384269s" totalImagesPullingTime="409.424316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:23.118103752 +0000 UTC m=+539.859799372" watchObservedRunningTime="2026-11-01 22:31:23.120384269 +0000 UTC m=+539.862079889"
Nov 01 22:31:23.503993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.503960    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"]
Nov 01 22:31:23.508112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.508084    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.521468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.520840    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-381659-serving-cert\""
Nov 01 22:31:23.526145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.526115    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"]
Nov 01 22:31:23.590338 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.590297    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:23.606757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.606723    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v4lc\" (UniqueName: \"kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.606982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.606859    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.606982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.606925    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.606982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.606960    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.708618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.708580    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v4lc\" (UniqueName: \"kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.708821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.708684    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.708821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.708722    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.708821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.708752    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.709479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.709451    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.711290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.711253    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.711416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.711292    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.720059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.720032    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v4lc\" (UniqueName: \"kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc\") pod \"maas-api-e2e-worker-w1-381659-66b885f674-qmgxd\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:23.722943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.722913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:23.835001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:23.834910    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:24.128243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.128211    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"]
Nov 01 22:31:24.319187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.319154    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"]
Nov 01 22:31:24.324068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.324036    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.329382 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.329348    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-3d8490-serving-cert\""
Nov 01 22:31:24.336673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.336644    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"]
Nov 01 22:31:24.418639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.418537    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldmgs\" (UniqueName: \"kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.418639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.418580    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.418928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.418720    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.418928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.418782    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.435259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.435216    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:24.519983 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.519949    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.520447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.520006    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.520447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.520062    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldmgs\" (UniqueName: \"kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.520447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.520080    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.520894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.520872    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.522585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.522565    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.522697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.522681    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.528679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.528657    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldmgs\" (UniqueName: \"kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs\") pod \"maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.560082 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.560035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:24.642633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.642601    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:24.931340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:24.931309    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"]
Nov 01 22:31:24.933379 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:31:24.933349    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2947187c_8bf2_42c3_9ac4_6b045474baa2.slice/crio-38a890e6ee2a0832ef89b60f5a10eaa580f07c79e07f989fc19a2017678742ed WatchSource:0}: Error finding container 38a890e6ee2a0832ef89b60f5a10eaa580f07c79e07f989fc19a2017678742ed: Status 404 returned error can't find the container with id 38a890e6ee2a0832ef89b60f5a10eaa580f07c79e07f989fc19a2017678742ed
Nov 01 22:31:25.111054 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.110958    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp" event={"ID":"2947187c-8bf2-42c3-9ac4-6b045474baa2","Type":"ContainerStarted","Data":"38a890e6ee2a0832ef89b60f5a10eaa580f07c79e07f989fc19a2017678742ed"}
Nov 01 22:31:25.112780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.112754    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd" event={"ID":"a27ee460-17f6-4b56-ae1e-8886c70a5da1","Type":"ContainerStarted","Data":"6c7f3d3c316738e0923321b69bb4e4b2f135600baa95504b7b9e8b0ecf991c2c"}
Nov 01 22:31:25.112913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.112796    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd" event={"ID":"a27ee460-17f6-4b56-ae1e-8886c70a5da1","Type":"ContainerStarted","Data":"d3315de19d08e371951f9df7cc23cc16b2900b7601ac80f5cfe9cc1b9d9db248"}
Nov 01 22:31:25.112913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.112900    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:25.135914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.135857    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd" podStartSLOduration=1.661404227 podStartE2EDuration="2.135840947s" totalImagesPullingTime="474.43672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:25.134333065 +0000 UTC m=+541.876028681" watchObservedRunningTime="2026-11-01 22:31:25.135840947 +0000 UTC m=+541.877536568"
Nov 01 22:31:25.267714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.267678    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:25.392895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.392854    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:25.972755 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:25.972719    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:26.120145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:26.120102    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp" event={"ID":"2947187c-8bf2-42c3-9ac4-6b045474baa2","Type":"ContainerStarted","Data":"eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7"}
Nov 01 22:31:26.120361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:26.120269    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:26.142486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:26.142423    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp" podStartSLOduration=1.7769666229999999 podStartE2EDuration="2.142405176s" totalImagesPullingTime="365.438553ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:26.140099943 +0000 UTC m=+542.881795565" watchObservedRunningTime="2026-11-01 22:31:26.142405176 +0000 UTC m=+542.884100797"
Nov 01 22:31:26.707178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:26.707147    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:27.378351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:27.378318    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:28.076084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:28.076043    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:28.684895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:28.684857    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:29.110255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:29.110222    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:31:29.482899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:29.482856    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:30.081679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:30.081642    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:30.343017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:30.342919    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:31:30.787953 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:30.787916    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:31.126256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:31.126177    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:31:31.585181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:31.585149    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:32.132485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:32.132455    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:31:32.465316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:32.465269    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:33.511799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:33.511754    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:34.643242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:34.643192    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:35.613058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:35.613018    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:36.587140 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:36.587099    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:36.975504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:36.975441    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:37.401922 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:37.401847    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:37.703406 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:37.703373    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:38.051269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:38.051187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:38.437005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:38.436896    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:38.818239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:38.818205    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:39.053248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:39.053206    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:39.672405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:39.672359    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:40.018967 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:31:40.018929    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:31:40.146374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:40.146319    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:40.846959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:40.846906    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:41.862939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:41.862901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:42.441903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:42.441867    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:31:42.504861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:42.504829    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:43.104763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:43.104729    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:43.238811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:43.238771    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:43.385650 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:43.385564    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:43.700188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:43.700153    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:43.932679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:43.932589    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:44.082816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:44.082732    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:44.419204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:44.419097    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:46.062028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.061992    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:46.146259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.146213    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"]
Nov 01 22:31:46.150845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.150820    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.158783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.158730    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e25182\"/\"openshift-service-ca.crt\""
Nov 01 22:31:46.159778 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.159757    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-e25182\"/\"kube-root-ca.crt\""
Nov 01 22:31:46.160098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.160081    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e25182\"/\"default-dockercfg-ln5ll\""
Nov 01 22:31:46.160287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.160272    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-e25182\"/\"simulator-w0-e25182-kserve-self-signed-certs\""
Nov 01 22:31:46.161140 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.161113    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"]
Nov 01 22:31:46.220476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.220436    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:46.239400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.239370    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.239622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.239423    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.239622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.239579    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.239757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.239638    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.239757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.239708    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26qw\" (UniqueName: \"kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.239757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.239739    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.340915    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.340983    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.341039    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.341074    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.341126    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26qw\" (UniqueName: \"kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.341153    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.341459    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.341668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.341485    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.342182 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.342161    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.344230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.344210    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.345021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.345001    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.349634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.349614    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26qw\" (UniqueName: \"kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw\") pod \"simulator-w0-e25182-kserve-7bb544d64b-wctg7\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") " pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.473658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.473615    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:46.745398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.745362    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:46.758607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.758470    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"]
Nov 01 22:31:46.760705 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:31:46.760675    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod01d597ea_da12_43c9_bd1a_dd26d47fd332.slice/crio-5226dd15018f77f7d6e3432e2eb027f9168a8dc72e96c5119bb19ab70e6383f0 WatchSource:0}: Error finding container 5226dd15018f77f7d6e3432e2eb027f9168a8dc72e96c5119bb19ab70e6383f0: Status 404 returned error can't find the container with id 5226dd15018f77f7d6e3432e2eb027f9168a8dc72e96c5119bb19ab70e6383f0
Nov 01 22:31:46.977698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:46.977645    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:47.240242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:47.240199    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" event={"ID":"01d597ea-da12-43c9-bd1a-dd26d47fd332","Type":"ContainerStarted","Data":"5226dd15018f77f7d6e3432e2eb027f9168a8dc72e96c5119bb19ab70e6383f0"}
Nov 01 22:31:47.670019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:47.669931    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:48.406198 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.406162    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:31:48.473007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.472974    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:48.764536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.764502    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"]
Nov 01 22:31:48.768821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.768795    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.778702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.778666    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"]
Nov 01 22:31:48.801454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.801417    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-00e49d\"/\"openshift-service-ca.crt\""
Nov 01 22:31:48.801637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.801433    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-00e49d\"/\"default-dockercfg-6hshw\""
Nov 01 22:31:48.801637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.801482    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-00e49d\"/\"kube-root-ca.crt\""
Nov 01 22:31:48.802453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.802227    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-00e49d\"/\"simulator-w3-00e49d-kserve-self-signed-certs\""
Nov 01 22:31:48.838780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.838741    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:48.866999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.866966    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.867178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.867005    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8flm\" (UniqueName: \"kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.867178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.867033    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.867178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.867164    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.867297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.867185    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.867297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.867202    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968131 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968091    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968156    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8flm\" (UniqueName: \"kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968201    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968246    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968270    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968287    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.968798 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.968773    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.969124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.969101    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.969476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.969446    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.971720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.971694    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.971921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.971902    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:48.977236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:48.977215    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8flm\" (UniqueName: \"kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm\") pod \"simulator-w3-00e49d-kserve-56f8f7b454-llkv9\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") " pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:49.103046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:49.102958    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:49.256220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:49.256179    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" event={"ID":"01d597ea-da12-43c9-bd1a-dd26d47fd332","Type":"ContainerStarted","Data":"382fe443abef6f43684af63e100466060796f0fb79583c3fb08e467e3ef2f292"}
Nov 01 22:31:49.276130 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:49.276070    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podStartSLOduration=1.763848621 podStartE2EDuration="3.276052794s" totalImagesPullingTime="1.512204173s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:49.273555594 +0000 UTC m=+566.015251213" watchObservedRunningTime="2026-11-01 22:31:49.276052794 +0000 UTC m=+566.017748413"
Nov 01 22:31:49.361794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:49.361701    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:49.402232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:49.402194    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"]
Nov 01 22:31:49.407454 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:31:49.407414    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddde04f9a_4461_49da_8aba_7bc3e19edb59.slice/crio-d237627a20ef51186ae423fa63c119e62d36de65df92624c0df5d0820a22bf0b WatchSource:0}: Error finding container d237627a20ef51186ae423fa63c119e62d36de65df92624c0df5d0820a22bf0b: Status 404 returned error can't find the container with id d237627a20ef51186ae423fa63c119e62d36de65df92624c0df5d0820a22bf0b
Nov 01 22:31:49.473836 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:49.473810    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:50.079002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:50.078938    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:50.198548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:50.198513    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:50.264672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:50.264629    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" event={"ID":"dde04f9a-4461-49da-8aba-7bc3e19edb59","Type":"ContainerStarted","Data":"96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12"}
Nov 01 22:31:50.264841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:50.264684    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" event={"ID":"dde04f9a-4461-49da-8aba-7bc3e19edb59","Type":"ContainerStarted","Data":"d237627a20ef51186ae423fa63c119e62d36de65df92624c0df5d0820a22bf0b"}
Nov 01 22:31:50.284268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:50.284216    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podStartSLOduration=2.284203416 podStartE2EDuration="2.284203416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:50.282082717 +0000 UTC m=+567.023778336" watchObservedRunningTime="2026-11-01 22:31:50.284203416 +0000 UTC m=+567.025899078"
Nov 01 22:31:50.873189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:50.873150    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:51.103426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:51.103380    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:51.478933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:51.478897    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:52.079304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.079275    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:52.763532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.763485    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"]
Nov 01 22:31:52.767801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.767778    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.775372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.775345    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-3d8490\"/\"openshift-service-ca.crt\""
Nov 01 22:31:52.775543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.775425    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-3d8490\"/\"kube-root-ca.crt\""
Nov 01 22:31:52.780658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.778408    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"]
Nov 01 22:31:52.780658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.778510    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d8490\"/\"default-dockercfg-8gvtb\""
Nov 01 22:31:52.780658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.778621    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d8490\"/\"simulator-w2-3d8490-kserve-self-signed-certs\""
Nov 01 22:31:52.801064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.801030    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:52.912615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.912583    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.912805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.912644    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttckf\" (UniqueName: \"kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.912805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.912749    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.912895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.912870    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.912935 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.912904    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.913037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.913011    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:52.941558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:52.941068    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:53.014096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014016    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014065    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014110    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014157    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014231    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttckf\" (UniqueName: \"kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014444 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014287    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014587    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014643    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.014716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.014677    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.016905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.016880    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.017117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.017099    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.024846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.024818    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttckf\" (UniqueName: \"kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf\") pod \"simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") " pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.085617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.085579    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:53.371659 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.371634    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"]
Nov 01 22:31:53.373458 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:31:53.373425    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5f3a24a7_886c_450f_b301_a9b83ec2f243.slice/crio-df6dc51e37d982a579413faa48e7fd00cde67d147536dc2e474ab3d7f26caa86 WatchSource:0}: Error finding container df6dc51e37d982a579413faa48e7fd00cde67d147536dc2e474ab3d7f26caa86: Status 404 returned error can't find the container with id df6dc51e37d982a579413faa48e7fd00cde67d147536dc2e474ab3d7f26caa86
Nov 01 22:31:53.725059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:53.725024    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:54.202568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.199660    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"]
Nov 01 22:31:54.218514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.218276    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"]
Nov 01 22:31:54.218678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.218543    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.227515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.223261    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-b28a40\"/\"simulator-w6-b28a40-kserve-self-signed-certs\""
Nov 01 22:31:54.227515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.223529    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-b28a40\"/\"default-dockercfg-6lslv\""
Nov 01 22:31:54.227515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.223956    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-b28a40\"/\"kube-root-ca.crt\""
Nov 01 22:31:54.227515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.224109    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-b28a40\"/\"openshift-service-ca.crt\""
Nov 01 22:31:54.273988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.273951    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:54.289674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.289639    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" event={"ID":"5f3a24a7-886c-450f-b301-a9b83ec2f243","Type":"ContainerStarted","Data":"5ecc203b041ff8dab8b023547762ffd0da631890a087922ec6ce095c1996e706"}
Nov 01 22:31:54.289840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.289686    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" event={"ID":"5f3a24a7-886c-450f-b301-a9b83ec2f243","Type":"ContainerStarted","Data":"df6dc51e37d982a579413faa48e7fd00cde67d147536dc2e474ab3d7f26caa86"}
Nov 01 22:31:54.311556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.311477    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podStartSLOduration=2.311457722 podStartE2EDuration="2.311457722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:54.308333906 +0000 UTC m=+571.050029527" watchObservedRunningTime="2026-11-01 22:31:54.311457722 +0000 UTC m=+571.053153342"
Nov 01 22:31:54.329775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.329742    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.329942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.329803    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.329942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.329855    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.329942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.329922    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.330127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.329951    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.330127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.329986    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gz5r\" (UniqueName: \"kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431110    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431197    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431245    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431323    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431357    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431393    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gz5r\" (UniqueName: \"kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.431770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.431741    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.432068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.432044    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.432180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.432158    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.434487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.434462    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.438008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.437957    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.444076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.443414    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gz5r\" (UniqueName: \"kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r\") pod \"simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") " pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.484145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.484058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:54.556239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.556202    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:54.836215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:54.836183    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"]
Nov 01 22:31:54.838860 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:31:54.838831    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb5c05fa8_2874_4cb3_ba0a_f570a02800a6.slice/crio-e085e65fd2a08dc87ce42109796fde76069850274eab252d8ced399b3696d6ee WatchSource:0}: Error finding container e085e65fd2a08dc87ce42109796fde76069850274eab252d8ced399b3696d6ee: Status 404 returned error can't find the container with id e085e65fd2a08dc87ce42109796fde76069850274eab252d8ced399b3696d6ee
Nov 01 22:31:55.085906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:55.085815    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:31:55.168086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:55.168034    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:55.297460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:55.297422    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" event={"ID":"b5c05fa8-2874-4cb3-ba0a-f570a02800a6","Type":"ContainerStarted","Data":"e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9"}
Nov 01 22:31:55.297882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:55.297473    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" event={"ID":"b5c05fa8-2874-4cb3-ba0a-f570a02800a6","Type":"ContainerStarted","Data":"e085e65fd2a08dc87ce42109796fde76069850274eab252d8ced399b3696d6ee"}
Nov 01 22:31:55.317172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:55.317128    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podStartSLOduration=1.317115934 podStartE2EDuration="1.317115934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:31:55.314564306 +0000 UTC m=+572.056259924" watchObservedRunningTime="2026-11-01 22:31:55.317115934 +0000 UTC m=+572.058811553"
Nov 01 22:31:55.556984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:55.556940    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:31:56.076372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:56.076337    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:56.473922 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:56.473890    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:56.488308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:56.488275    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:56.758664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:56.758581    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:57.333700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:57.333655    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:31:57.441840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:57.441803    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:58.066402 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:58.066365    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:58.772099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:58.772059    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:59.104047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:59.103967    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:59.117036 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:59.117009    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:59.352163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:59.352131    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:31:59.483416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:59.483385    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:31:59.815772 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:31:59.815684    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:32:00.169806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:00.169714    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:00.980509 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:00.980434    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:01.668431 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:01.668395    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:02.291374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:02.291322    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:02.873078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:02.872985    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:03.086410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:03.086376    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:32:03.100048 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:03.100023    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:32:03.284302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:03.284262    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:03.387650 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:03.387622    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:32:03.792304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:03.792270    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:04.550823 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:04.550783    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:04.556388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:04.556361    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:32:04.569813 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:04.569788    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:32:05.372980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:05.372938    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:05.403145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:05.403121    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:32:06.089996 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:06.089964    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:06.848709 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:06.848670    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:07.457470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:07.457431    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:08.142520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:08.142462    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:08.757524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:08.757474    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:09.468820 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:09.468789    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:10.015544 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:32:10.015508    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:32:10.125054 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:10.125016    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:10.951799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:10.951760    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:11.024345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:11.024299    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:32:11.764561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:11.764521    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:12.391711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:12.391672    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:12.512388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:12.512344    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:13.061395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:13.061357    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:13.399981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:13.399891    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:13.777472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:13.777439    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:14.573544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:14.573515    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:15.371723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:15.371691    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:16.164728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:16.164695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:16.960902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:16.960865    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:17.882277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:17.882242    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:18.627250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:18.627206    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:19.267814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:19.267769    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:19.869653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:19.869608    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:20.541831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.541787    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:20.860921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.860834    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"]
Nov 01 22:32:20.868681 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.868653    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.880991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.880966    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-00e49d\"/\"premium-w3-00e49d-kserve-self-signed-certs\""
Nov 01 22:32:20.882794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.882768    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"]
Nov 01 22:32:20.933797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.933759    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.933979 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.933810    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.933979 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.933926    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.933979 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.933972    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.934113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.934085    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.934149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.934116    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4lg4\" (UniqueName: \"kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:20.936467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:20.936424    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:21.035309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.035266    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.035547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.035327    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.035547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.035405    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.035547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.035450    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.035547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.035533    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.035782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.035566    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4lg4\" (UniqueName: \"kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.036142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.036098    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.036273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.036170    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.036443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.036423    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.038763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.038740    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.039150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.039123    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.043454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.043427    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4lg4\" (UniqueName: \"kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4\") pod \"premium-w3-00e49d-kserve-7dd8b4d787-7lm8c\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") " pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.183368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.183270    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:21.464072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.464041    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:21.472922 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.472889    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"]
Nov 01 22:32:21.480026 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:32:21.477301    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod04f907c0_418a_4a90_8d9f_204203d366d4.slice/crio-5e03b1bb05da7a64fffea4d7bfe6ccd20ec0807c0ebcdf5cdc8761110eb5fdf7 WatchSource:0}: Error finding container 5e03b1bb05da7a64fffea4d7bfe6ccd20ec0807c0ebcdf5cdc8761110eb5fdf7: Status 404 returned error can't find the container with id 5e03b1bb05da7a64fffea4d7bfe6ccd20ec0807c0ebcdf5cdc8761110eb5fdf7
Nov 01 22:32:21.492775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:21.492711    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" event={"ID":"04f907c0-418a-4a90-8d9f-204203d366d4","Type":"ContainerStarted","Data":"5e03b1bb05da7a64fffea4d7bfe6ccd20ec0807c0ebcdf5cdc8761110eb5fdf7"}
Nov 01 22:32:22.222401 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:22.222365    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:22.502077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:22.501971    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" event={"ID":"04f907c0-418a-4a90-8d9f-204203d366d4","Type":"ContainerStarted","Data":"6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6"}
Nov 01 22:32:22.518830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:22.518775    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podStartSLOduration=2.518760119 podStartE2EDuration="2.518760119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:32:22.517212843 +0000 UTC m=+599.258908462" watchObservedRunningTime="2026-11-01 22:32:22.518760119 +0000 UTC m=+599.260455738"
Nov 01 22:32:22.902616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:22.902512    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:23.119201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:23.119147    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:23.184601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:23.184478    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:23.447073 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:23.446997    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:32:23.448487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:23.448459    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:32:23.650039 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:23.650008    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:24.271933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.271891    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:24.410802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.410759    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"]
Nov 01 22:32:24.417462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.417428    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.428724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.426706    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d8490\"/\"premium-w2-3d8490-kserve-self-signed-certs\""
Nov 01 22:32:24.433874 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.431677    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"]
Nov 01 22:32:24.498984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.498951    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:24.578814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.578715    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.579157 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.579126    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.579352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.579318    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.579544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.579528    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.579749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.579735    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.579944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.579928    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx5ns\" (UniqueName: \"kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.681305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.681261    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx5ns\" (UniqueName: \"kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.681512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.681431    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.681512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.681473    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.681512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.681510    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.681774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.681545    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.681960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.681928    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.682089 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.682054    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.682212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.682195    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.682381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.682349    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.684816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.684791    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.684907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.684799    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.689411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.689379    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx5ns\" (UniqueName: \"kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns\") pod \"premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") " pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:24.745660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:24.745623    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:25.032575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.032235    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"]
Nov 01 22:32:25.034804 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:32:25.034773    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf32326fc_3d31_4930_9150_da39082bc0d4.slice/crio-d2a359c080ceed7e27af987742f1c302b8b0529bfa1608d86c75263c610aec44 WatchSource:0}: Error finding container d2a359c080ceed7e27af987742f1c302b8b0529bfa1608d86c75263c610aec44: Status 404 returned error can't find the container with id d2a359c080ceed7e27af987742f1c302b8b0529bfa1608d86c75263c610aec44
Nov 01 22:32:25.077559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.077529    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:25.520381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.520339    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" event={"ID":"f32326fc-3d31-4930-9150-da39082bc0d4","Type":"ContainerStarted","Data":"c81667e1c4e11bc62e632f257c89ffe7b34d93334643f95317fb3365d79975ad"}
Nov 01 22:32:25.520876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.520393    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" event={"ID":"f32326fc-3d31-4930-9150-da39082bc0d4","Type":"ContainerStarted","Data":"d2a359c080ceed7e27af987742f1c302b8b0529bfa1608d86c75263c610aec44"}
Nov 01 22:32:25.538194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.538142    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podStartSLOduration=1.538127984 podStartE2EDuration="1.538127984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:32:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:32:25.535687103 +0000 UTC m=+602.277382726" watchObservedRunningTime="2026-11-01 22:32:25.538127984 +0000 UTC m=+602.279823597"
Nov 01 22:32:25.746262 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.746222    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:25.853298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:25.853205    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:26.193760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:26.193678    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:26.645741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:26.645702    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:27.099656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.099616    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:32:27.312487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.312454    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:27.337962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.337795    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"]
Nov 01 22:32:27.343789 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.343761    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.353479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.353391    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"]
Nov 01 22:32:27.353694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.353672    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-b28a40\"/\"premium-w6-b28a40-kserve-self-signed-certs\""
Nov 01 22:32:27.436189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.436147    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:27.514347 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.514303    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.514570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.514428    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs6l2\" (UniqueName: \"kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.514570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.514526    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.514965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.514607    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.514965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.514741    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.514965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.514768    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.615867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.615781    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.615876    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.615896    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.615949    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.615986    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs6l2\" (UniqueName: \"kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.616042    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.616777    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.616909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.616818    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.617097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.616951    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.621513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.619763    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.621513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.620013    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.624013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.623985    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs6l2\" (UniqueName: \"kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2\") pod \"premium-w6-b28a40-kserve-74544b57bf-pjn4s\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") " pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.677986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.677953    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:27.957804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:27.957777    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"]
Nov 01 22:32:27.959632 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:32:27.959602    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61bbc377_4c9a_46b8_b8d7_05bbe87be152.slice/crio-3dd5ca4e7814453d1f9438e58377b7866edec17358337fbb04ce5b5955655be7 WatchSource:0}: Error finding container 3dd5ca4e7814453d1f9438e58377b7866edec17358337fbb04ce5b5955655be7: Status 404 returned error can't find the container with id 3dd5ca4e7814453d1f9438e58377b7866edec17358337fbb04ce5b5955655be7
Nov 01 22:32:28.058233 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:28.057776    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:28.545720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:28.545679    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" event={"ID":"61bbc377-4c9a-46b8-b8d7-05bbe87be152","Type":"ContainerStarted","Data":"97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13"}
Nov 01 22:32:28.545877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:28.545730    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" event={"ID":"61bbc377-4c9a-46b8-b8d7-05bbe87be152","Type":"ContainerStarted","Data":"3dd5ca4e7814453d1f9438e58377b7866edec17358337fbb04ce5b5955655be7"}
Nov 01 22:32:28.573476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:28.573409    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podStartSLOduration=1.573388875 podStartE2EDuration="1.573388875s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:32:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:32:28.571051399 +0000 UTC m=+605.312747020" watchObservedRunningTime="2026-11-01 22:32:28.573388875 +0000 UTC m=+605.315084495"
Nov 01 22:32:28.678615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:28.678573    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:28.972773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:28.972737    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:29.756251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:29.756207    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:30.384020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:30.383990    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:31.073199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:31.073164    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:31.183665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:31.183624    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:31.196904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:31.196876    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:31.584377 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:31.584348    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:32:31.830193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:31.830146    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:32.543434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:32.543401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:33.274974 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:33.274939    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:33.979774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:33.979739    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:34.746151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:34.746110    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:34.764714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:34.764671    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:34.765111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:34.765084    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:35.459533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:35.459485    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:35.603668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:35.603637    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:32:36.262851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:36.262814    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:37.275926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:37.275527    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:37.679207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:37.679133    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:37.692702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:37.692674    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:38.181867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:38.181824    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:38.625343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:38.625312    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:32:39.131168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:39.131126    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:39.787928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:39.787875    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:40.014994 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:32:40.014960    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:32:40.644516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:40.644450    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:41.284698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:41.284663    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:41.953772 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:41.953738    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:43.154514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:43.147274    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:43.872948 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:43.872901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:44.618618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:44.618579    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:45.354947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:45.354909    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:45.982599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:45.982555    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:46.104110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:46.104074    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:32:46.798902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:46.798861    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:47.713338 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:47.713291    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:48.498988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:48.498942    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:49.173251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:49.173209    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:49.762855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:49.762810    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:50.367718 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:50.367675    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:51.011937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:51.011901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:51.748676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:51.748607    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:52.374283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:52.374248    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:53.069679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:53.069641    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:53.801346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:53.801311    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:54.389521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:54.389455    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:54.976029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:54.975991    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:55.674444 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:55.674408    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:56.271437 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:56.271401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:57.062211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:57.062183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:57.981421 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:57.981388    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:58.930803 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:58.930765    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:32:59.252302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.249108    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"]
Nov 01 22:32:59.258616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.258584    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.269529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.269477    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"]
Nov 01 22:32:59.309411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.309385    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-b28a40-openshift-default-dockercfg-mwllr\""
Nov 01 22:32:59.368942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.368901    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.368960    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369067    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369095    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369135    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369201    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84dlm\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369271    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369303    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.369395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.369339    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470039 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470001    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84dlm\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470056    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470092    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470128    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470221    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470257    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470314    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470341    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470394    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.470946 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.470901    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.471043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.471013    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.471107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.471031    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.471316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.471288    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.471419 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.471357    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.472783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.472753    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.472998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.472982    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.495806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.495784    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.497745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.497724    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84dlm\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.575160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.575071    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:32:59.862791 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.862729    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"]
Nov 01 22:32:59.865912 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:32:59.865881    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbdcb39_f084_4626_9541_d5b0be671b8d.slice/crio-bd9b317db6f217778f68d0205fb3a78977e556b981de37438b6652db153d7808 WatchSource:0}: Error finding container bd9b317db6f217778f68d0205fb3a78977e556b981de37438b6652db153d7808: Status 404 returned error can't find the container with id bd9b317db6f217778f68d0205fb3a78977e556b981de37438b6652db153d7808
Nov 01 22:32:59.868374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.868340    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:32:59.868463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.868422    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:32:59.868567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:32:59.868469    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:00.132685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:00.132584    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:00.753517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:00.753330    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" event={"ID":"0cbdcb39-f084-4626-9541-d5b0be671b8d","Type":"ContainerStarted","Data":"20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3"}
Nov 01 22:33:00.753517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:00.753396    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" event={"ID":"0cbdcb39-f084-4626-9541-d5b0be671b8d","Type":"ContainerStarted","Data":"bd9b317db6f217778f68d0205fb3a78977e556b981de37438b6652db153d7808"}
Nov 01 22:33:00.776702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:00.776634    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podStartSLOduration=1.776616432 podStartE2EDuration="1.776616432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:00.775219061 +0000 UTC m=+637.516914683" watchObservedRunningTime="2026-11-01 22:33:00.776616432 +0000 UTC m=+637.518312052"
Nov 01 22:33:01.060965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:01.060879    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:01.575555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:01.575520    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:33:01.577438 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:01.577410    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 01 22:33:01.577579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:01.577469    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 01 22:33:01.886266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:01.886173    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:02.576193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:02.576158    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 01 22:33:02.576647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:02.576222    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 01 22:33:02.801469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:02.801434    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:03.480412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:03.480374    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:03.575644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:03.575608    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 01 22:33:03.575850 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:03.575668    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 01 22:33:04.310450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:04.310415    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:04.575557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:04.575454    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 01 22:33:04.575557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:04.575538    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 01 22:33:05.181654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:05.181622    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:05.575670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:05.575628    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Nov 01 22:33:05.576075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:05.575696    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Nov 01 22:33:05.865812 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:05.865715    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:06.486874 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:06.486838    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:06.580189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:06.580160    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:33:06.580601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:06.580588    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:33:06.581161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:06.581142    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:33:07.191175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:07.191142    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:07.956274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:07.956230    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:08.002443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:08.002404    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:33:08.782015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:08.781983    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:09.471924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:09.471886    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:10.014665 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:33:10.014626    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:33:10.452232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:10.452196    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:11.079690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:11.079654    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:11.856108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:11.856070    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:12.484296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:12.484264    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:12.602312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:12.602275    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:13.274089 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:13.274053    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:13.975720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:13.975685    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:14.770136 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:14.770099    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:15.466271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:15.466230    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:16.166169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:16.166130    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:16.898618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:16.898587    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:17.574185 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:17.574147    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:18.182020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:18.181982    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:18.961283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:18.961236    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:19.577674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:19.577616    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:20.367887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:20.367851    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:21.092213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:21.092173    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:21.690420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:21.690384    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:22.357991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:22.357953    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:22.975749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:22.975711    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:23.772224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:23.772192    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:24.566613 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:24.566575    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:25.272272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:25.272233    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:25.929764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:25.929734    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:26.584907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:26.584870    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:27.363785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:27.363747    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:27.567752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:27.567722    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:27.680757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:27.680667    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:28.032485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.032452    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:28.046585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.046546    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"]
Nov 01 22:33:28.052357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.052320    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.061249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.061220    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"]
Nov 01 22:33:28.065525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.064194    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-aa441b\"/\"default-dockercfg-hvjd5\""
Nov 01 22:33:28.067841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.067618    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-aa441b\"/\"distinct-w5-aa441b-kserve-self-signed-certs\""
Nov 01 22:33:28.069554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.069535    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-aa441b\"/\"kube-root-ca.crt\""
Nov 01 22:33:28.069836 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.069806    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-aa441b\"/\"openshift-service-ca.crt\""
Nov 01 22:33:28.167587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.167549    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:28.185591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.183451    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.185591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.183557    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.185591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.183584    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2hfn\" (UniqueName: \"kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.185591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.183641    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.185591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.183669    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.185591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.183717    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.284658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.284578    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.284658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.284650    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.284875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.284675    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2hfn\" (UniqueName: \"kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.284875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.284724    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.284875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.284750    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.284875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.284793    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.285202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.285182    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.285526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.285432    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.286008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.285985    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.287643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.287618    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.287756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.287707    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.293369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.293347    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2hfn\" (UniqueName: \"kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn\") pod \"distinct-w5-aa441b-kserve-5456b84c89-8dtnd\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") " pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.372220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.372186    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:28.656785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.656743    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"]
Nov 01 22:33:28.658253 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:28.658220    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2c13e54_c14f_46c5_bd5d_03414f2534b1.slice/crio-6408c67e35cc1fb29313da758b8b54e160ca5837fb982436dfbb83ceaaf0792d WatchSource:0}: Error finding container 6408c67e35cc1fb29313da758b8b54e160ca5837fb982436dfbb83ceaaf0792d: Status 404 returned error can't find the container with id 6408c67e35cc1fb29313da758b8b54e160ca5837fb982436dfbb83ceaaf0792d
Nov 01 22:33:28.759762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.758919    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:28.920409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.920322    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" event={"ID":"d2c13e54-c14f-46c5-bd5d-03414f2534b1","Type":"ContainerStarted","Data":"c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e"}
Nov 01 22:33:28.920409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.920372    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" event={"ID":"d2c13e54-c14f-46c5-bd5d-03414f2534b1","Type":"ContainerStarted","Data":"6408c67e35cc1fb29313da758b8b54e160ca5837fb982436dfbb83ceaaf0792d"}
Nov 01 22:33:28.937720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:28.937569    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podStartSLOduration=0.937551893 podStartE2EDuration="937.551893ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:28.936902386 +0000 UTC m=+665.678598007" watchObservedRunningTime="2026-11-01 22:33:28.937551893 +0000 UTC m=+665.679247513"
Nov 01 22:33:29.372416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:29.372388    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:29.662013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:29.661929    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:30.034621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:30.034581    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:33:30.395887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:30.395785    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:31.350311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:31.350180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:32.086584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:32.086546    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:33.170568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:33.167207    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:33.971723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:33.971692    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:34.772079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:34.772044    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:35.855478 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:35.855398    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:36.597631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:36.597597    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:37.185947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.185891    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"]
Nov 01 22:33:37.190551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.190528    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.202124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.202096    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"]
Nov 01 22:33:37.212534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.207596    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-3d8490\"/\"distinct-w2-3d8490-kserve-self-signed-certs\""
Nov 01 22:33:37.224456 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.224420    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.224785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.224740    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.224910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.224792    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.224910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.224834    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhbhq\" (UniqueName: \"kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.224910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.224876    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.225087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.224912    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.264889 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.264852    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.325637    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.325698    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.325744    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhbhq\" (UniqueName: \"kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.325789    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.325828    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.325954    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.326515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.326397    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.327061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.326917    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.330511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.327441    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.330511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.330276    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.339878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.339818    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhbhq\" (UniqueName: \"kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.348961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.348879    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm\") pod \"distinct-w2-3d8490-kserve-685c9d87d5-hrhn7\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") " pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.512212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.512170    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:37.713379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.713245    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:37.837819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.837778    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"]
Nov 01 22:33:37.839295 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:37.839266    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0b83430_f00d_4de0_a667_28278090f818.slice/crio-14efd7874407d0ad9aa339155458e50ae43b17af403e59c74d23c5147b2d353f WatchSource:0}: Error finding container 14efd7874407d0ad9aa339155458e50ae43b17af403e59c74d23c5147b2d353f: Status 404 returned error can't find the container with id 14efd7874407d0ad9aa339155458e50ae43b17af403e59c74d23c5147b2d353f
Nov 01 22:33:37.996346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.996305    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" event={"ID":"d0b83430-f00d-4de0-a667-28278090f818","Type":"ContainerStarted","Data":"6145270f009e847ea2b66a55889ebd7ad832cd16a4b517d0d3a0c6c888bc3a1e"}
Nov 01 22:33:37.996586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:37.996357    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" event={"ID":"d0b83430-f00d-4de0-a667-28278090f818","Type":"ContainerStarted","Data":"14efd7874407d0ad9aa339155458e50ae43b17af403e59c74d23c5147b2d353f"}
Nov 01 22:33:38.018105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:38.018047    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podStartSLOduration=1.018029645 podStartE2EDuration="1.018029645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:38.016665856 +0000 UTC m=+674.758361474" watchObservedRunningTime="2026-11-01 22:33:38.018029645 +0000 UTC m=+674.759725264"
Nov 01 22:33:38.373592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:38.373556    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:38.402177 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:38.402130    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:38.513729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:38.513695    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:38.803446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:38.803401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:39.017814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:39.017780    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:33:39.729634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:39.729376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:39.888517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:39.888455    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:40.069587 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:33:40.067114    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:33:40.803067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:40.802942    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:41.658435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:41.658390    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:42.389341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:42.389271    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:43.082407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.082366    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:43.802379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.802345    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"]
Nov 01 22:33:43.806683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.806655    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.816394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.816340    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b42cc4-openshift-default-dockercfg-lt2tw\""
Nov 01 22:33:43.818733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.818705    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"]
Nov 01 22:33:43.866830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.866798    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:43.889455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889418    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889458    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gfhc2\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889557    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889597    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889626    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889674    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889693    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889824    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.889906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.889888    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.968010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.967965    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"]
Nov 01 22:33:43.971856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.971832    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.975362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.975335    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-381659-openshift-default-dockercfg-wjt66\""
Nov 01 22:33:43.991041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991007    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"]
Nov 01 22:33:43.991286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991113    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991216    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991250    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991283    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991347    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991408    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991441    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991458    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gfhc2\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991483    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991532    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991548    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991566    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991598    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991617    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmfst\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991642    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991662    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991680    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.991897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.991699    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:43.992823 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.992793    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.993037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.993012    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.993162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.993088    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.993162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.993144    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.993381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.993359    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:43.994672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:43.994645    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:44.001061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.001025    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gfhc2\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:44.001061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.001058    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:44.001969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.001943    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:44.092610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092527    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092583    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092618    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092635    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmfst\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092658    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092683    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092719    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092757    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.092848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.092809    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.093427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.093399    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.093555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.093462    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.093689 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.093670    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.093733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.093714    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.093805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.093782    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.095419 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.095399    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.095780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.095761    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.099977 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.099952    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.100093 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.100033    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmfst\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-db6mw\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.122542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.122501    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:44.229642 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.229608    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"]
Nov 01 22:33:44.233805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.233778    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.241717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.241685    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-e25182-openshift-default-dockercfg-pf4mc\""
Nov 01 22:33:44.256886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.256835    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"]
Nov 01 22:33:44.285829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.285790    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:44.294212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294170    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294277    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294322    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwvms\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294343    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294363    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294423    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294635    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294675    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.294809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.294778    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395275    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395327    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395386    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395529    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395587    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395636    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwvms\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395668    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395714    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395756    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.395860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395759    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.396079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.395975    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.396253 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.396230    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.396515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.396470    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.397158 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.397044    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.401944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.401922    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.402911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.402872    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.405030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.405008    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.405434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.405240    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwvms\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.414138 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.414114    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"]
Nov 01 22:33:44.416735 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:44.416710    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed73a15_8dd4_451b_8110_46a3f73c22c2.slice/crio-d77693612dce29e7eb838c438e0eb47fdfd1e4340f7da60fd183336e155dffd6 WatchSource:0}: Error finding container d77693612dce29e7eb838c438e0eb47fdfd1e4340f7da60fd183336e155dffd6: Status 404 returned error can't find the container with id d77693612dce29e7eb838c438e0eb47fdfd1e4340f7da60fd183336e155dffd6
Nov 01 22:33:44.420058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.420013    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.420160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.420111    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.420160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.420146    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.550331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.550290    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:44.581256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.581190    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"]
Nov 01 22:33:44.584016 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:44.583980    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82a6301_b1dc_4116_9f89_95dc2ccec308.slice/crio-929cb46a19e1850746480366464b2b36622921fe8a04cd07b1fde8e367d69915 WatchSource:0}: Error finding container 929cb46a19e1850746480366464b2b36622921fe8a04cd07b1fde8e367d69915: Status 404 returned error can't find the container with id 929cb46a19e1850746480366464b2b36622921fe8a04cd07b1fde8e367d69915
Nov 01 22:33:44.589607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.589569    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.589710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.589645    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.589710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.589677    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.888415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.888326    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:44.896684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.896531    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"]
Nov 01 22:33:44.900506 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:44.900453    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c8c0f55_2c7d_45fc_85ff_ac07c88f07e1.slice/crio-b6d3e54f27321d931d3685167f12a64a2ad5bd35da14457d9d360e3b3b10388f WatchSource:0}: Error finding container b6d3e54f27321d931d3685167f12a64a2ad5bd35da14457d9d360e3b3b10388f: Status 404 returned error can't find the container with id b6d3e54f27321d931d3685167f12a64a2ad5bd35da14457d9d360e3b3b10388f
Nov 01 22:33:44.903815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.903778    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.903936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.903849    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:44.903936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:44.903880    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:45.051413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.051373    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" event={"ID":"fed73a15-8dd4-451b-8110-46a3f73c22c2","Type":"ContainerStarted","Data":"e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437"}
Nov 01 22:33:45.051624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.051446    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" event={"ID":"fed73a15-8dd4-451b-8110-46a3f73c22c2","Type":"ContainerStarted","Data":"d77693612dce29e7eb838c438e0eb47fdfd1e4340f7da60fd183336e155dffd6"}
Nov 01 22:33:45.055589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.055558    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" event={"ID":"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1","Type":"ContainerStarted","Data":"1c6b7db0b161849223c1daa9e2e541840c3f55bd86bb97ce6dd5481fb58b1d8e"}
Nov 01 22:33:45.055805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.055790    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" event={"ID":"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1","Type":"ContainerStarted","Data":"b6d3e54f27321d931d3685167f12a64a2ad5bd35da14457d9d360e3b3b10388f"}
Nov 01 22:33:45.059281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.059253    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" event={"ID":"c82a6301-b1dc-4116-9f89-95dc2ccec308","Type":"ContainerStarted","Data":"bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516"}
Nov 01 22:33:45.059393 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.059308    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" event={"ID":"c82a6301-b1dc-4116-9f89-95dc2ccec308","Type":"ContainerStarted","Data":"929cb46a19e1850746480366464b2b36622921fe8a04cd07b1fde8e367d69915"}
Nov 01 22:33:45.077063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.076995    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podStartSLOduration=2.076974617 podStartE2EDuration="2.076974617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:45.074816627 +0000 UTC m=+681.816512242" watchObservedRunningTime="2026-11-01 22:33:45.076974617 +0000 UTC m=+681.818670239"
Nov 01 22:33:45.094747 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.094677    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podStartSLOduration=1.094658016 podStartE2EDuration="1.094658016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:45.092343852 +0000 UTC m=+681.834039473" watchObservedRunningTime="2026-11-01 22:33:45.094658016 +0000 UTC m=+681.836353637"
Nov 01 22:33:45.114307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.114247    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podStartSLOduration=2.114213719 podStartE2EDuration="2.114213719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:45.110199111 +0000 UTC m=+681.851894756" watchObservedRunningTime="2026-11-01 22:33:45.114213719 +0000 UTC m=+681.855909320"
Nov 01 22:33:45.122886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.122842    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:45.124553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.124486    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:45.124750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.124721    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:45.287789 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.287750    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:45.289886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.289856    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:45.290118 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.290083    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:45.560575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.556548    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:45.560759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.560610    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:45.560759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.560683    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:45.771516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:45.770943    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:46.126514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:46.125026    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:46.126514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:46.125087    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:46.290513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:46.289929    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:46.290513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:46.289989    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:46.554519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:46.552903    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:46.554519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:46.552964    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:47.003517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.002933    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:33:47.127518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.124966    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:47.127518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.125041    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:47.290033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.289936    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:47.290283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.290256    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:47.533610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.533574    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:47.557919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.557681    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:47.557919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.557745    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:47.561515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.558754    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:47.811575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:47.811363    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:48.126815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.126721    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:48.127081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.127055    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:48.289065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.289031    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:48.289649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.289616    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:48.382328 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.382226    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:33:48.530971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.530938    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:33:48.551074 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.551034    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:48.551366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:48.551322    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:49.151586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.145035    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:49.151586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.145093    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:49.292300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.292263    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:49.292994 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.292946    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:49.380940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.380905    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:49.387515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.384404    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:49.387515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.385001    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:33:49.422574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.410694    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:33:49.586032 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.574822    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:49.586032 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:49.574885    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:50.013565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.006676    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:33:50.135515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.134632    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:50.135515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.135272    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:50.135515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.135315    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:50.291577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.290920    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:50.291577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.290987    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:50.322518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.319649    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:50.386518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.385577    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:50.456519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.455612    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:50.456519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.456235    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:33:50.502515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.497767    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:33:50.563574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.554580    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:50.563574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.554641    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:50.840175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:50.840080    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:33:51.130568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.129623    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:51.130568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.129682    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:51.291518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.290567    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:51.291518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.290633    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:51.376514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.374172    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:51.553524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.552922    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:51.553524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:51.552980    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:52.125520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.122923    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Nov 01 22:33:52.125520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.122990    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Nov 01 22:33:52.299228 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.297924    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:52.299228 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.297976    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:52.491516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.491183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:52.550919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.550883    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Nov 01 22:33:52.551188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.551145    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Nov 01 22:33:52.872381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.871737    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"]
Nov 01 22:33:52.877792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.877763    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.881915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.881883    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"]
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965149    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcs88\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965207    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965234    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965263    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965323    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965434    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965480    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965582    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:52.978380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:52.965614    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.067480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.067434    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.067805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.067773    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.067990    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068136 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068121    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068262 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068247    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcs88\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068377 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068364    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068481    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068648    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068826    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.068995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.068968    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.069486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.069272    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.069486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.069383    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.069785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.069684    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.069883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.069810    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.072294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.072257    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.072588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.072567    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.078373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.078346    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcs88\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.080650 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.080629    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-wl9g5\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.204752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.204719    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:53.289064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.289021    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body=
Nov 01 22:33:53.289292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.289111    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused"
Nov 01 22:33:53.674938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.674826    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"]
Nov 01 22:33:53.714162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.711747    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:53.728301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.728028    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:53.728301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.728129    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:53.728301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.728181    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:53.979374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:53.976915    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:54.169440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.165604    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:33:54.169440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.165674    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:54.277912 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.277816    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" event={"ID":"58ff371d-d2ed-415c-988f-cffea5d08b83","Type":"ContainerStarted","Data":"b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7"}
Nov 01 22:33:54.278171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.278146    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" event={"ID":"58ff371d-d2ed-415c-988f-cffea5d08b83","Type":"ContainerStarted","Data":"77f92e4f2db7fa78a758619441a9b5169ff19c0ce2214297828f695567e05a5d"}
Nov 01 22:33:54.318618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.318047    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podStartSLOduration=2.318028148 podStartE2EDuration="2.318028148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:33:54.316798991 +0000 UTC m=+691.058494611" watchObservedRunningTime="2026-11-01 22:33:54.318028148 +0000 UTC m=+691.059723771"
Nov 01 22:33:54.639568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.638673    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:33:54.639568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.638750    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:54.680523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.677047    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:54.681003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.680664    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:54.906644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:54.904756    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:33:55.056935 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.056836    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:55.226571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.226538    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:33:55.252519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.230307    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:33:55.252519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.243707    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:33:55.292957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.292924    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:33:55.293263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.293232    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:33:55.311734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.311698    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:55.312692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.312627    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:55.314685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.314670    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:33:55.509712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.509625    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:55.510847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.510822    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:55.580742 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:55.580711    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:33:56.156888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:56.156838    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:56.205774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:56.205739    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:33:56.205974 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:56.205795    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:33:57.092318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:57.092271    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:57.205672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:57.205637    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:33:57.206135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:57.205694    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:33:57.970866 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:57.970830    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:58.206072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.206030    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:33:58.206571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.206093    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:33:58.791847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.791814    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:58.903936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.903897    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"]
Nov 01 22:33:58.907780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.907756    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:58.912426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.912404    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-00e49d-openshift-default-dockercfg-xhjdm\""
Nov 01 22:33:58.942067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:58.942033    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"]
Nov 01 22:33:59.012991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.012952    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwjcd\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013026    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013081    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013113    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013192    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013238    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013315    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013336    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.013431 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.013360    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.094280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.094186    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"]
Nov 01 22:33:59.098203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.098179    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.112755 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.112727    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-aa441b-openshift-default-dockercfg-kjt95\""
Nov 01 22:33:59.113697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.113671    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"]
Nov 01 22:33:59.114314 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114294    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114324    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114373    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114399    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114750    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114774    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114804    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.114983 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.114863    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.115064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.115043    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwjcd\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.115223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.115202    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.115386 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.115371    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.115452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.115403    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.115452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.115422    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.115452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.115422    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.149722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.117657    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.154558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.153797    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.158438 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.158409    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.160172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.160141    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwjcd\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.203188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.203147    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"]
Nov 01 22:33:59.205445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.205417    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:33:59.205617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.205465    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:33:59.208601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.208582    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.213187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.213163    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-3d8490-openshift-default-dockercfg-vp9kv\""
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.247731    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.248929    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpdf\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.258673    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.258797    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.258887    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.258970    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.259051    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.259105    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.259153    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.269678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.259390    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.275067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.275030    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"]
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362598    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362638    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362656    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362687    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362706    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362726    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362760    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362813    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362841    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362856    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362902    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpdf\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362933    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362950    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.362978    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.363000    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.363022    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.363045    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.363336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.363069    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqqg8\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.364413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.363728    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.364413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.363899    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.365151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.365064    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.365313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.365173    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.365313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.365214    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.367763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.367218    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.400254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.369928    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.400254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.373785    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.400254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.375120    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpdf\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.416522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.416477    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:33:59.465237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.465197    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.465452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.465433    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqqg8\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.465883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.465861    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.466410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.466393    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.466841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.466750    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.467412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.466234    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.467412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.467012    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.467592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.467288    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.468086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.468062    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.468184 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.468111    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.468247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.468186    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.468982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.468959    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.469452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.469431    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.469816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.469791    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.471534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.471512    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.471812 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.471786    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.476583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.476559    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.477715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.477673    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqqg8\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.576834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.574992    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:33:59.596432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.596393    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"]
Nov 01 22:33:59.598114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.598080    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.598200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.598145    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.598200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.598177    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.810620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.810580    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"]
Nov 01 22:33:59.834603 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:59.834562    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b34e89_e83e_406d_8c6b_8589cbd6fd63.slice/crio-02cfef4fc38f7601917746322fb52e31d406bed8627365c2b1be185d65cbcfbc WatchSource:0}: Error finding container 02cfef4fc38f7601917746322fb52e31d406bed8627365c2b1be185d65cbcfbc: Status 404 returned error can't find the container with id 02cfef4fc38f7601917746322fb52e31d406bed8627365c2b1be185d65cbcfbc
Nov 01 22:33:59.854512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.854385    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.854512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.854483    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.854735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.854551    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.971816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.968153    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"]
Nov 01 22:33:59.972735 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:33:59.972708    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1b24ec1_6153_4575_83b2_4f894618dd72.slice/crio-5b8c92fb9872fdd0bced41be6ebd26805653f5b07b28bdc9c64d6e7b6fecf0f1 WatchSource:0}: Error finding container 5b8c92fb9872fdd0bced41be6ebd26805653f5b07b28bdc9c64d6e7b6fecf0f1: Status 404 returned error can't find the container with id 5b8c92fb9872fdd0bced41be6ebd26805653f5b07b28bdc9c64d6e7b6fecf0f1
Nov 01 22:33:59.983788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.975999    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:33:59.991506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.989053    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.991506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.989147    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:33:59.991506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:33:59.989192    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:00.208529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.205289    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:34:00.208529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.205350    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:34:00.366518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.364808    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" event={"ID":"c1b24ec1-6153-4575-83b2-4f894618dd72","Type":"ContainerStarted","Data":"239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426"}
Nov 01 22:34:00.366518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.364901    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" event={"ID":"c1b24ec1-6153-4575-83b2-4f894618dd72","Type":"ContainerStarted","Data":"5b8c92fb9872fdd0bced41be6ebd26805653f5b07b28bdc9c64d6e7b6fecf0f1"}
Nov 01 22:34:00.373543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.372784    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" event={"ID":"0be1d300-18b4-47c5-b74d-05bce19f3411","Type":"ContainerStarted","Data":"ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6"}
Nov 01 22:34:00.373852 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.373809    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" event={"ID":"0be1d300-18b4-47c5-b74d-05bce19f3411","Type":"ContainerStarted","Data":"149b5ae805a3f6ee3226c34597f1aa31d34ff8b5b850a028a3693147455c27c5"}
Nov 01 22:34:00.381248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.381217    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" event={"ID":"94b34e89-e83e-406d-8c6b-8589cbd6fd63","Type":"ContainerStarted","Data":"5eb7672754f6255ed3a3c1c143773e5663da15479ed124fc7787c8942aef6353"}
Nov 01 22:34:00.381524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.381465    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" event={"ID":"94b34e89-e83e-406d-8c6b-8589cbd6fd63","Type":"ContainerStarted","Data":"02cfef4fc38f7601917746322fb52e31d406bed8627365c2b1be185d65cbcfbc"}
Nov 01 22:34:00.391457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.391406    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podStartSLOduration=1.391394117 podStartE2EDuration="1.391394117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:00.385749671 +0000 UTC m=+697.127445290" watchObservedRunningTime="2026-11-01 22:34:00.391394117 +0000 UTC m=+697.133089736"
Nov 01 22:34:00.417078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.417005    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:34:00.418538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.418266    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podStartSLOduration=1.41824824 podStartE2EDuration="1.41824824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:00.416868857 +0000 UTC m=+697.158564477" watchObservedRunningTime="2026-11-01 22:34:00.41824824 +0000 UTC m=+697.159943861"
Nov 01 22:34:00.421754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.418828    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:00.421754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.418893    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:00.470622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.469598    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podStartSLOduration=2.469573224 podStartE2EDuration="2.469573224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:33:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:00.45473528 +0000 UTC m=+697.196430914" watchObservedRunningTime="2026-11-01 22:34:00.469573224 +0000 UTC m=+697.211268843"
Nov 01 22:34:00.590673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:00.590629    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:34:01.207896 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.207857    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:34:01.208203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.208173    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:34:01.229515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.226011    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:01.264087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.264056    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:34:01.264461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.264369    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:01.264658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.264633    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:01.435969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.419256    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:01.435969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.419317    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:01.581569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.581019    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:01.581569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:01.581079    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:02.212519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.211923    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:34:02.212519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.211987    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:34:02.237518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.236596    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:02.264516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.262901    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:02.264516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.262961    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:02.414517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.410592    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:02.422210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.422171    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:02.422457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.422427    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:02.582027 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.580966    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:02.582027 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.581034    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:02.582027 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.581429    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:02.694776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:02.694729    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.010519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.009588    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.102705 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.102642    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.206075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.206040    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body=
Nov 01 22:34:03.206408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.206343    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused"
Nov 01 22:34:03.224671 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.224635    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:03.258006 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.257967    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:03.258663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.258628    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:03.348775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.331596    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.450767 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.450731    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:03.451106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.451071    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:03.583102 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.583050    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.584112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.584088    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:34:03.585783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.585757    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:03.585974 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.585951    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:03.698564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.695714    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.698564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.696938    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:03.855518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.854720    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:03.958580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:03.947111    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:04.018744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.018696    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:04.020054 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.019964    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:34:04.095648 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.095603    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:04.189514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.177608    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:34:04.189514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.178991    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:34:04.286568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.285895    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:04.286568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.285961    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:04.354184 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.354127    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:04.511134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.432983    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:04.511134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.433055    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:04.511134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.435093    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:04.588387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.576330    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:04.588387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.576393    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:04.636518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.635619    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:04.699660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.698589    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:04.869574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.869028    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:04.875311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:04.870552    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:34:05.075518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.075014    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:05.101521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.095562    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:05.163727 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.160597    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:34:05.185430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.185396    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:05.281285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.281248    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:34:05.281562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.281527    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:05.282922 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.282896    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:05.283086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.283063    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:05.439071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.438968    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:05.441015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.440983    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:05.441210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.441187    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:05.595517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.589722    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:05.612618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.612582    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:05.612885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.612857    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:05.649516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.642848    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:05.682236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.682202    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:34:05.869624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:05.869582    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:06.114910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.114868    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:34:06.148698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.148600    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:06.157519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.156609    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:06.157895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.157857    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:06.284516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.280823    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:06.284516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.280882    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:06.322525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.318396    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:06.360635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.359791    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:06.421230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.421141    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:06.530668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.430744    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:34:06.560536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.461800    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:06.560536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.530560    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-wl9g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:34:06.560536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.531559    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:06.561784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.561748    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:06.679566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.678579    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:06.679566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.679436    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:34:06.737309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.737273    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:06.737603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.737572    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:06.783610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.781379    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:34:06.808095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:06.808063    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:07.229518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.223836    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:34:07.229518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.225073    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:34:07.250517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.242846    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 22:34:07.276516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.276456    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:07.276720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.276532    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:07.361518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.360586    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:07.361518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.361359    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:34:07.410515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.410103    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:34:07.456532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.450585    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:07.456532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.450644    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:07.498830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.498660    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:07.579031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.578995    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:07.579682 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.579645    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:07.679647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.679602    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:07.681013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.680990    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:34:07.841157 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:07.841066    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:08.118371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.118287    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:34:08.148637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.148590    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:08.149174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.122677    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:08.154677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.119864    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:08.258517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.258420    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Nov 01 22:34:08.258731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.258612    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Nov 01 22:34:08.372576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.369617    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:08.392318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.392287    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:34:08.422929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.422889    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Nov 01 22:34:08.423305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.423277    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Nov 01 22:34:08.586526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.586006    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Nov 01 22:34:08.586526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.586080    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Nov 01 22:34:08.863554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:08.863523    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:09.676413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.676373    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:34:09.678646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.678625    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:34:09.679528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.679472    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:34:09.797361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.797329    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:34:09.798266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.798207    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:34:09.801183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.800956    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:34:09.864235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.864202    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:09.881461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.881428    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:34:09.888987 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.888959    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:34:09.922281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:09.903459    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:34:10.017847 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:34:10.017812    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:34:10.481561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:10.477146    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:10.895130 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:10.895044    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:11.322698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:11.322615    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:11.445630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:11.441637    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:12.006590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:12.004190    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:12.553960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:12.553917    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:12.986879 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:12.986834    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:13.899805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:13.899730    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:14.249953 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.249920    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"]
Nov 01 22:34:14.254707 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.254680    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.281835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.281798    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"]
Nov 01 22:34:14.404671 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404637    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.404671 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404675    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.404887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404696    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46hxm\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.404887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404788    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.404887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404867    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.404980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404900    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.404980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.404954    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.405043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.405004    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.405101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.405078    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506041    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506087    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506103    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46hxm\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506125    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506199    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506238    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506337    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506362    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506407    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506762    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.506855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.506809    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.507046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.507013    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.507168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.507027    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.507168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.507089    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.508542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.508518    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.509105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.509080    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.513934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.513907    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.514202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.514182    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-46hxm\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.579640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.579604    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:14.655212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.655041    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:14.894783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.894756    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"]
Nov 01 22:34:14.896335 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:34:14.896309    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c6d58e_2db3_4e3f_a2ae_3917cf8b52a3.slice/crio-73f540e53b4193f180762861ae2de3cb2d69ea835ce50e96029698d962877971 WatchSource:0}: Error finding container 73f540e53b4193f180762861ae2de3cb2d69ea835ce50e96029698d962877971: Status 404 returned error can't find the container with id 73f540e53b4193f180762861ae2de3cb2d69ea835ce50e96029698d962877971
Nov 01 22:34:14.898899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.898855    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:14.898990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.898941    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:14.898990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:14.898976    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:15.594297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:15.594256    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:15.709149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:15.709107    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:15.884162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:15.884067    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" event={"ID":"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3","Type":"ContainerStarted","Data":"6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a"}
Nov 01 22:34:15.884409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:15.884388    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" event={"ID":"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3","Type":"ContainerStarted","Data":"73f540e53b4193f180762861ae2de3cb2d69ea835ce50e96029698d962877971"}
Nov 01 22:34:15.909602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:15.909544    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podStartSLOduration=1.909524 podStartE2EDuration="1.909524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:15.907862406 +0000 UTC m=+712.649558024" watchObservedRunningTime="2026-11-01 22:34:15.909524 +0000 UTC m=+712.651219621"
Nov 01 22:34:16.480208 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:16.480176    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:16.580203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:16.580174    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:16.581989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:16.581961    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 22:34:16.582103 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:16.582017    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 22:34:17.259845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:17.259806    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:17.580970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:17.580888    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 22:34:17.580970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:17.580943    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 22:34:18.043802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.043766    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:18.580369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.580321    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 22:34:18.580817 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.580396    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 22:34:18.780634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.780599    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"]
Nov 01 22:34:18.786785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.786751    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"]
Nov 01 22:34:18.786961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.786867    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:18.801588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.800048    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-381659\"/\"default-dockercfg-x4b79\""
Nov 01 22:34:18.801588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.800136    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-381659\"/\"distinct-w1-381659-kserve-self-signed-certs\""
Nov 01 22:34:18.803626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.803600    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-381659\"/\"openshift-service-ca.crt\""
Nov 01 22:34:18.811167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.811145    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-381659\"/\"kube-root-ca.crt\""
Nov 01 22:34:18.889646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.889530    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:18.915539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.913408    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:18.955518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.954749    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:18.955518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.954802    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:18.955518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.954859    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kks9f\" (UniqueName: \"kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:18.955518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.954916    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:18.955518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:18.954976    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.066795    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.066906    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.067066    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.067093    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.067140    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kks9f\" (UniqueName: \"kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.067181    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.067514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.067396    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:19.068009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.067778    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.074035 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.068342    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.074035 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.069361    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.089522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.077456    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.094582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.091303    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.098151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.098096    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kks9f\" (UniqueName: \"kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f\") pod \"distinct-w1-381659-kserve-6bb4786f94-jxx4q\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") " pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.117232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.117201    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:19.514839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.514809    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"]
Nov 01 22:34:19.516502 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:34:19.516454    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0e94f7e_58f3_407e_b3c1_7aac0082c4b6.slice/crio-38e1ac3ecaeb7bd9899d53a0e22c2f154e0ff2df17d814cab2a86700850f39c0 WatchSource:0}: Error finding container 38e1ac3ecaeb7bd9899d53a0e22c2f154e0ff2df17d814cab2a86700850f39c0: Status 404 returned error can't find the container with id 38e1ac3ecaeb7bd9899d53a0e22c2f154e0ff2df17d814cab2a86700850f39c0
Nov 01 22:34:19.580133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.580093    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 22:34:19.580296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.580179    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 22:34:19.801199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.801101    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:19.964539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.964470    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" event={"ID":"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6","Type":"ContainerStarted","Data":"a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5"}
Nov 01 22:34:19.964720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.964553    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" event={"ID":"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6","Type":"ContainerStarted","Data":"38e1ac3ecaeb7bd9899d53a0e22c2f154e0ff2df17d814cab2a86700850f39c0"}
Nov 01 22:34:19.982951 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:19.982896    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podStartSLOduration=1.982882704 podStartE2EDuration="1.982882704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:19.980520153 +0000 UTC m=+716.722215773" watchObservedRunningTime="2026-11-01 22:34:19.982882704 +0000 UTC m=+716.724578346"
Nov 01 22:34:20.122577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:20.121587    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:20.516146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:20.516108    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:20.580790 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:20.580755    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 22:34:20.581016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:20.580827    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 22:34:21.232668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:21.230131    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:21.580634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:21.580544    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Nov 01 22:34:21.580634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:21.580609    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Nov 01 22:34:21.988978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:21.988941    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:22.584997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:22.584960    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:22.585473 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:22.585351    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:22.586477 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:22.586459    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:34:22.757844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:22.757803    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:23.555815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:23.555777    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:24.275937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:24.275894    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:25.003070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:25.003023    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:25.675626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:25.675587    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:26.403155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:26.403121    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:27.188558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:27.188525    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:27.709580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:27.709538    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:28.131025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:28.130948    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:34:28.197655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:28.197612    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:29.117837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:29.117803    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:29.143477 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:29.143446    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:29.255931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:29.255893    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:30.741938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:30.741903    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:31.048572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.047816    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:31.076091 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.076053    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:31.147480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.147436    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:31.319674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.319575    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:31.320543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.319902    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:31.402276 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.402234    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:31.657779 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:31.657651    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:32.089538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.082075    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:32.206543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.143525    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:32.206543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.143838    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:32.206543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.148059    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:32.206543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.148345    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:32.206543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.205783    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:32.395773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.391070    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:32.427297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.427267    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:32.451804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:32.451770    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:33.560316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:33.560279    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:34.269199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:34.269157    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:35.176970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:35.176937    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:35.855219 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:35.855185    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:36.480520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:36.480458    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:37.181902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.181870    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:37.227880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.227834    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"]
Nov 01 22:34:37.232275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.232248    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.239797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.239771    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b42cc4\"/\"openshift-service-ca.crt\""
Nov 01 22:34:37.240891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.240849    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b42cc4\"/\"kube-root-ca.crt\""
Nov 01 22:34:37.241014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.241002    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b42cc4\"/\"distinct-2-w4-b42cc4-kserve-self-signed-certs\""
Nov 01 22:34:37.241127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.241094    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b42cc4\"/\"default-dockercfg-gczbq\""
Nov 01 22:34:37.244570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.244543    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"]
Nov 01 22:34:37.312029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.311983    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:37.334780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.334736    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhl9d\" (UniqueName: \"kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.334991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.334846    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.334991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.334881    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.335119 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.335057    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.335176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.335130    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.335176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.335163    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.436792    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.436890    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.436950    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.436983    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhl9d\" (UniqueName: \"kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.437093    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.437122    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.440439    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.440741    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.443256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.441205    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.480247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.480196    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.481506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.481467    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhl9d\" (UniqueName: \"kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.498517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.494577    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm\") pod \"distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") " pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.565257 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.565224    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:37.877509 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:37.877462    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"]
Nov 01 22:34:37.881070 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:34:37.881036    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod858b7e22_cc93_4c35_b03f_fd8c833ede79.slice/crio-53a2c4884d02205fa3e098949306a71bcf15edf5ba95c34f6a2ba2cdd2d5074d WatchSource:0}: Error finding container 53a2c4884d02205fa3e098949306a71bcf15edf5ba95c34f6a2ba2cdd2d5074d: Status 404 returned error can't find the container with id 53a2c4884d02205fa3e098949306a71bcf15edf5ba95c34f6a2ba2cdd2d5074d
Nov 01 22:34:38.136873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:38.136785    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:38.283427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:38.283384    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" event={"ID":"858b7e22-cc93-4c35-b03f-fd8c833ede79","Type":"ContainerStarted","Data":"fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03"}
Nov 01 22:34:38.283427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:38.283437    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" event={"ID":"858b7e22-cc93-4c35-b03f-fd8c833ede79","Type":"ContainerStarted","Data":"53a2c4884d02205fa3e098949306a71bcf15edf5ba95c34f6a2ba2cdd2d5074d"}
Nov 01 22:34:38.301190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:38.301138    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podStartSLOduration=1.301124832 podStartE2EDuration="1.301124832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:38.299686005 +0000 UTC m=+735.041381624" watchObservedRunningTime="2026-11-01 22:34:38.301124832 +0000 UTC m=+735.042820450"
Nov 01 22:34:38.565872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:38.565837    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:38.962564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:38.962514    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:39.675788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:39.675743    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:40.016526 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:34:40.016471    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:34:40.471873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:40.471833    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:41.259792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:41.259758    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:41.876876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:41.876834    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:42.665153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:42.665115    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:43.275949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:43.275912    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:43.900144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:43.900101    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"]
Nov 01 22:34:43.904329 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:43.904309    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:43.939688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:43.939663    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"]
Nov 01 22:34:43.993821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:43.993789    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:44.031428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031386    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031444    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031598    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031641    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031756    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g4pc\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031843    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031873    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.031919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031910    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.032013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.031934    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.075284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.075245    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"]
Nov 01 22:34:44.079837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.079809    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.092232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.092205    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"]
Nov 01 22:34:44.107314 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.107279    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"]
Nov 01 22:34:44.111611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.111588    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138222    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138284    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138329    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138359    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138394    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138421    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138459    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138532    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138561    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138592    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138656    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138689    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138736    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138793    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138823    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138860    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmxcm\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138911    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.138975    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g4pc\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.139639    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.139892    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.140359    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.141014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.140623    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.142237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.141562    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"]
Nov 01 22:34:44.144213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.144184    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.165396 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.165326    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.174823 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.174792    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.175658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.175612    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.176134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.176079    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g4pc\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.240016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.239979    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.240201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240049    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.240201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240080    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.240201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240158    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.240397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240204    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.240397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240239    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.240397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240266    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.240397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240311    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.240397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240336    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mmvf\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.240397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240384    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.240740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240411    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.240740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240626    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:44.240929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240909    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.241000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.240964    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.241295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241118    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.241295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241160    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.241463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241369    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.241463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241441    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmxcm\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.241598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241521    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.241598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241574    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.241708 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241625    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.241869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.241829    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.242535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.242050    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.242535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.242478    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.245300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.244257    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.249939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.246129    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.249939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.247394    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"]
Nov 01 22:34:44.252343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.252326    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.266578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.266550    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"]
Nov 01 22:34:44.269700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.269661    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.275531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.274578    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmxcm\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-9nkgt\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.279544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.278628    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"]
Nov 01 22:34:44.290592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.290083    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.302472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.301239    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"]
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344103    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344169    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344206    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344243    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344309    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344333    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344383    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344408    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsx97\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344447    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344477    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344544    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344570    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mmvf\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344594    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344620    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344664    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344694    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344732    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344778    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344863    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344889    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344944    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.344984    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.345015    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.345046    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dt67\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.345075    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.345099    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.348881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.345172    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.360080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.356001    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.360080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.356823    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.360080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.356874    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.360080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.357181    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.360080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.357311    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.360080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.360005    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.361249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.361196    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.368415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.368394    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.369216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.369193    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mmvf\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf\") pod \"e2e-worker-w1-381659-openshift-default-894db6755-q6ldq\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") " pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.398596 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.398308    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:44.411143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.411115    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"]
Nov 01 22:34:44.425549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.419687    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.441963 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.439164    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.452393    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453239    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453342    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453400    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453479    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453535    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453606    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsx97\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453652    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453714    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453758    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453805    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453876    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453942    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.454039    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.454143    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.454177    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.454225    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dt67\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.454255    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.453111    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.456248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.455126    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.460166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.458947    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.460166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.459385    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.460166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.459649    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.460166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.459670    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.460166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.459900    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.460166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.460112    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.464065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.464042    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.464364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.464342    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.464546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.464459    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"]
Nov 01 22:34:44.467924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.467857    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.469236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.469184    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.470193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.470171    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.470894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.470820    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.470894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.470824    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsx97\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.472515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.472436    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dt67\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.472698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.472657    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.474339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.474299    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token\") pod \"e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") " pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.556664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.556393    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.556664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.556455    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.556914    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.556995    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.557107    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.557151    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.557179    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.557245    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.558264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.557341    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xnwd\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.612150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.612110    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"]
Nov 01 22:34:44.615370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.615337    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.615567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.615548    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.615661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.615650    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.626114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.625900    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:44.657094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.656710    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658302    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658386    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xnwd\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658442    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658482    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658536    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658586    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658671    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658710    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.658739    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.659451    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.661012 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.660893    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.661152 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.661129    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.662149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.661829    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.662149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.662105    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.662899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.662865    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.669245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.668864    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xnwd\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.669903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.669751    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.670414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.670382    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.790079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.790004    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"]
Nov 01 22:34:44.790666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.790379    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:44.793773 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:34:44.792586    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9cb7b00_e3a7_4264_a7c3_34b53ca35d84.slice/crio-ae02e60f104644f13fc17c4d5534d659f32d984c7c89432b238b55bdb65a99f2 WatchSource:0}: Error finding container ae02e60f104644f13fc17c4d5534d659f32d984c7c89432b238b55bdb65a99f2: Status 404 returned error can't find the container with id ae02e60f104644f13fc17c4d5534d659f32d984c7c89432b238b55bdb65a99f2
Nov 01 22:34:44.802074 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.800774    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"]
Nov 01 22:34:44.806643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.805533    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.806643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.805620    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.806643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.805660    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.817304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.812737    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.817304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.812812    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:44.817304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:44.812852    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.100797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.095998    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:45.135095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.134130    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.135095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.134227    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.135095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.134272    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.163320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.135423    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.166210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.166159    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.166773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.166687    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"]
Nov 01 22:34:45.166968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.166569    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.174390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.174314    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"]
Nov 01 22:34:45.245923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.245482    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:45.266207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.266133    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"]
Nov 01 22:34:45.276068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.274030    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.276068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.274114    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.276068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.274159    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:34:45.378085 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.378050    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" event={"ID":"031e1687-4ecd-4efb-b046-da95c18d4cb8","Type":"ContainerStarted","Data":"42fc1daa88c8568e816bc46897c1f2bf444ed5ce164a38b3100d00d2a28a67fb"}
Nov 01 22:34:45.380396 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.380365    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" event={"ID":"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84","Type":"ContainerStarted","Data":"b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1"}
Nov 01 22:34:45.380634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.380614    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" event={"ID":"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84","Type":"ContainerStarted","Data":"ae02e60f104644f13fc17c4d5534d659f32d984c7c89432b238b55bdb65a99f2"}
Nov 01 22:34:45.387867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.387842    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" event={"ID":"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6","Type":"ContainerStarted","Data":"472e99993a11170a6d0288884e565d0cebfc909cecbf96cefce8f863baafc601"}
Nov 01 22:34:45.395431 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.395404    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" event={"ID":"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5","Type":"ContainerStarted","Data":"ab373175abd37833e556a9fbe70c5b06bd5300c72c3bf6acd1dd6b7c044bc49c"}
Nov 01 22:34:45.395767 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.395743    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" event={"ID":"56baaa56-760a-4802-90ed-116711c2304d","Type":"ContainerStarted","Data":"9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6"}
Nov 01 22:34:45.410998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.410941    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podStartSLOduration=1.410922341 podStartE2EDuration="1.410922341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:45.404935545 +0000 UTC m=+742.146631165" watchObservedRunningTime="2026-11-01 22:34:45.410922341 +0000 UTC m=+742.152617961"
Nov 01 22:34:45.467517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.432284    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:45.467517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.466372    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" event={"ID":"56baaa56-760a-4802-90ed-116711c2304d","Type":"ContainerStarted","Data":"2ca842d240e6b1d566f3cc23fce4802fc251d1c172e980d792d01cce0ad9f218"}
Nov 01 22:34:45.467517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.466424    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:45.467517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.466459    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" event={"ID":"b9ecdaf9-8526-430b-9373-ec36c6897253","Type":"ContainerStarted","Data":"715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947"}
Nov 01 22:34:45.467517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.466555    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" event={"ID":"b9ecdaf9-8526-430b-9373-ec36c6897253","Type":"ContainerStarted","Data":"ec5a7ed2a57f53c20cad397ab44e7bbb4ef7c2ba22161e539fd9f105b5a9401d"}
Nov 01 22:34:45.470993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.469742    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:45.470993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.469790    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:45.474288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.474151    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podStartSLOduration=1.474134179 podStartE2EDuration="1.474134179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:45.470210155 +0000 UTC m=+742.211905776" watchObservedRunningTime="2026-11-01 22:34:45.474134179 +0000 UTC m=+742.215829800"
Nov 01 22:34:45.493265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:45.493209    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podStartSLOduration=2.49319433 podStartE2EDuration="2.49319433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:45.49024266 +0000 UTC m=+742.231938280" watchObservedRunningTime="2026-11-01 22:34:45.49319433 +0000 UTC m=+742.234889949"
Nov 01 22:34:46.216278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.216242    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:46.263085 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.262604    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:46.263441 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.263415    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:46.263689 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.263650    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:46.406694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.406655    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:46.406906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.406717    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:46.446551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.441776    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:46.446551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.441833    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:46.503187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.502042    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" event={"ID":"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6","Type":"ContainerStarted","Data":"5d8f2f116104ed5228cf1488be54754faee08fee77d47ffb0d59e46121848566"}
Nov 01 22:34:46.579518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.565792    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podStartSLOduration=2.565766607 podStartE2EDuration="2.565766607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:46.548821503 +0000 UTC m=+743.290517147" watchObservedRunningTime="2026-11-01 22:34:46.565766607 +0000 UTC m=+743.307462228"
Nov 01 22:34:46.657521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.640628    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:46.662569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.661901    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:46.662569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.661985    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:46.744515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.744426    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" event={"ID":"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5","Type":"ContainerStarted","Data":"bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8"}
Nov 01 22:34:46.779573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.758324    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" event={"ID":"031e1687-4ecd-4efb-b046-da95c18d4cb8","Type":"ContainerStarted","Data":"2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32"}
Nov 01 22:34:46.803632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.803595    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:46.803956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.803930    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:46.804115 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.804092    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:46.826668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.815383    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podStartSLOduration=2.815362373 podStartE2EDuration="2.815362373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:46.780166455 +0000 UTC m=+743.521862075" watchObservedRunningTime="2026-11-01 22:34:46.815362373 +0000 UTC m=+743.557057992"
Nov 01 22:34:46.875518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:46.864810    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podStartSLOduration=2.864789208 podStartE2EDuration="2.864789208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:34:46.801344021 +0000 UTC m=+743.543039666" watchObservedRunningTime="2026-11-01 22:34:46.864789208 +0000 UTC m=+743.606484834"
Nov 01 22:34:47.244969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.244928    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:47.245594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.245558    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:47.428417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.428379    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:47.428749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.428718    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:47.442520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.441320    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:47.443568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.443540    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:47.443763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.443742    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:47.570648 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.569845    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:34:47.632995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.632954    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:47.633281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.633251    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:47.643356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.643314    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:47.659975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.659934    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:47.662031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.661995    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:47.662261 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.662236    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:47.795024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.794988    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:47.795270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.795245    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:47.804675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.804634    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:47.837260 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:47.837111    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:48.060736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.060686    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.084719 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.083578    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:48.124638 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.116663    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.126923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.124957    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.241959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.241479    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:48.241959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.241580    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:48.316365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.316322    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.318936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.318903    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.401576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.400954    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:48.401576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.401013    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:48.448516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.447927    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:48.448516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.447991    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:48.509517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.501624    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.509517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.502388    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:48.604888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.604583    5057 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 01 22:34:48.642842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.642808    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:48.727533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.706999    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:48.727533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.704588    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:48.727533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.647603    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:48.727533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.707366    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:34:48.727533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.706943    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:48.727533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.708116    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:48.824517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.823800    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:48.824517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:48.823858    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.093571    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.093995    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.100884    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.101166    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.139904    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.141398    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.142427    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:34:49.161636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.142487    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:34:49.324714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.320708    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:49.324714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.322075    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:34:49.324714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.323167    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:34:49.324714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.323227    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:34:49.353532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.338252    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:49.353532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.338304    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:49.434318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.432172    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:49.434318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.432232    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:49.434318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.433557    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:49.481636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.481600    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:49.509060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.508929    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:49.509472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.509447    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:49.767521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.745574    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:49.808221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.805745    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:49.808221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.805810    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:49.808221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.805907    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:49.808221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.805932    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:49.808221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.808120    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:49.808221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.808163    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:49.873264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.855888    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:49.892448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:49.892412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:50.362655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.307000    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:50.362655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.338769    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:50.362655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.339515    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:34:50.392695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.365816    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:50.520541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.443388    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:50.520541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.443456    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:50.567309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.543270    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:34:50.655569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.594837    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.115518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.655903    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.154853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.625477    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.155384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.155352    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:51.155617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.677647    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.155801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.677910    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.156009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.873450    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.156207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.940140    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:51.156390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.155185    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:34:51.156586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.677365    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": context deadline exceeded"
Nov 01 22:34:51.156757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.683042    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.156900 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.156881    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:51.157044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.732369    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.157163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.157145    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:51.157283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.752689    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.157394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.157378    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:51.157551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:50.870392    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.157699 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.157673    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:51.196515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.185363    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:34:51.268649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.268612    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.268914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.268883    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:51.391305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.391213    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:51.444520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.401767    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.444520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.401828    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:51.444520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.402641    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.444520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.402917    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.444520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.403148    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:51.491657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.491617    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.491907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.491878    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:51.596518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.549239    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:51.655194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.570772    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:34:51.661520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.658399    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.665599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.665559    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:51.687535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.666797    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.687535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.666843    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:51.793628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.793590    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:51.793932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:51.793903    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:52.086629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.086579    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.146673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.146641    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:34:52.160086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.157643    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.160086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.157969    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.160086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.159283    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.247280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.247245    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:52.247585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.247541    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:52.401991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.401890    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:52.403244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.403212    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.404249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.404225    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:34:52.405204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.405169    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.406198 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.402614    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:52.442008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.441972    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:52.442313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.442268    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:52.504872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.504819    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:52.631913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.631874    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:52.632154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.632125    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:52.659047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.658932    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:34:52.659653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.659316    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:52.660355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.660333    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:52.794516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.784792    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:34:52.833517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.815900    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:52.833517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:52.815963    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:53.173983 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.152948    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.175346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.159889    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.175346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.159923    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:34:53.175346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.159945    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:53.175346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.160291    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.257213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.257091    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:53.257213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.257151    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:53.410346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.410288    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.410975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.410940    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.411391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.410384    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:53.411659 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.411634    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:53.455766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.455732    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:53.456008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.455977    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:53.506351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.506309    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.507003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.506977    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:34:53.517191 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.517163    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:53.632856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.632821    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:53.633121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.633088    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:53.660246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.660206    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:53.661446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.661416    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:53.661661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.661638    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:53.710554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.710450    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:53.819759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.819723    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:53.979044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:53.978939    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:54.174433 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.174386    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:54.181672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.181626    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:54.182688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.182654    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:34:54.183047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.183027    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:54.184553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.184531    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:34:54.184872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.184856    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:34:54.278571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.277469    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:54.278571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.277571    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:54.318099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.293371    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:54.491057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.491021    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:54.491677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.491611    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:54.493060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.493027    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:54.535547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.532050    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:54.536269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.536238    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:54.559851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.537709    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:54.560207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.537817    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:54.561753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.560162    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:54.651348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.651315    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:54.651634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.651605    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:54.677000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.676945    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:54.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.677307    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:54.716736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.716699    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:54.812037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.797458    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:54.812037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:54.810923    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:55.130318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.130229    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:34:55.132337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.132311    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:55.179675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.179618    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:55.195127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.190152    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:55.195127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.190464    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:34:55.195127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.190727    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:55.247529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.243834    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:55.247529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.243893    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:55.302095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.302061    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:55.475516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.460862    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:55.475516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.466749    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:55.496512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.492853    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:55.496512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.493149    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:55.555517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.497823    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:55.595178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.594977    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:34:55.595178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.595053    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:34:55.595178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.595099    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:55.624881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.624735    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded"
Nov 01 22:34:55.648516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.629923    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:34:55.671123 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.671074    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:55.671384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.671349    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:55.671949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.671927    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:55.672116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.672093    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:55.758445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.758339    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:55.804427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.804393    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:55.804793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:55.804750    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:56.239517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.217192    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:34:56.259574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.241985    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:56.259574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.243194    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:56.259574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.244736    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:56.259574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.244775    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:56.259574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.217934    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:56.424362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.423907    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:56.424362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.423967    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:56.442761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.442710    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:56.443057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.443008    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:56.549269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.549168    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:56.602899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.602845    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:56.603516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.603465    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:56.603975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.603951    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:34:56.604337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.604314    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:34:56.630413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.630381    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:56.630676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.630646    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:56.667454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.667422    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:56.667736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.667708    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:56.803582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.793344    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:56.803855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:56.803820    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:57.155724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.150241    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:34:57.170410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.165209    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.234512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.233597    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.234512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.234332    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:34:57.261632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.261588    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.264831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.261892    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.264831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.261919    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.342514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.323167    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:57.342514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.323227    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:57.512135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.489691    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:57.556181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.556129    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:57.609555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.606525    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:57.609555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.606976    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.609555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.607332    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.609982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.609956    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:34:57.611702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.611676    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:57.660926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.613980    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:57.660926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.660834    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:57.660926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.631035    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:57.660926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.660890    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:57.661434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.660782    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:57.674517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.663077    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:57.683598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.683557    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:57.706514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.702187    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:34:57.812664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.799891    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:57.812957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:57.812918    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:58.170893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.170762    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:58.172095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.172072    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:34:58.236711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.236011    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:58.253883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.242359    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:58.253883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.242417    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:58.271526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.267560    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:58.271526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.269113    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:58.271526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.269381    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:58.418002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.417964    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:58.418288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.418245    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:58.440355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.439894    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:58.440355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.439958    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:58.559516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.526209    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:58.636612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.563373    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:34:58.646518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.599757    5057 prober.go:122] "Probe failed" probeType="Startup" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 01 22:34:58.646518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.610789    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:34:58.646518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.610853    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.610883    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.610935    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.634315    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.673849    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.675581    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.675631    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:58.701576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.685603    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:58.911891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.911855    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:58.961543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:58.961440    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:59.131524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.118154    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"]
Nov 01 22:34:59.165283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.165250    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.174609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.174571    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:59.253199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.253156    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.303890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.303853    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"]
Nov 01 22:34:59.305075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.305043    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmjkb\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.317614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.317580    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.317826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.317802    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.317987 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.317969    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.318111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.318097    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.318370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.318348    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.318622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.318602    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.318784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.318765    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.318997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.318978    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.378952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.378917    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:34:59.379773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.379737    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:59.386523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.380204    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:34:59.399643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.399601    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:34:59.427873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.417124    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.427873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.417458    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.427873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.417764    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.442128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.437887    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:34:59.448905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.448866    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:34:59.474584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.474436    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.483745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.483708    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.483990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.483971    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.484209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.484194    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.484415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.484398    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmjkb\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.484540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.484523    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.505643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.505606    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.505882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.505859    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.506076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.506051    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.506427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.506397    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.513610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.505406    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.513610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.507255    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:34:59.513610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.507301    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:34:59.521522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.514776    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.522013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.521987    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.558515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.534102    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.640937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.640887    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:59.645516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.641934    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.697517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.692574    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.894629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.699578    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:34:59.896369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.699609    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:34:59.897563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.706621    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:34:59.897763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.897737    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:34:59.897880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.751716    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.897999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.788149    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.898106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.889628    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"]
Nov 01 22:34:59.910567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.751323    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmjkb\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.910767 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.751398    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:34:59.914518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.910892    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:34:59.914676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.913791    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"]
Nov 01 22:34:59.925923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.925898    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:34:59.933947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.933927    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:34:59.934103 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.934089    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:34:59.934245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.934232    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:34:59.934355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.934342    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:34:59.947033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.796602    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:34:59.952636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.952604    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:34:59.959599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.949432    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:34:59.959816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.913883    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:34:59.996522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.993236    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrvs9\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.005518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:34:59.996890    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.005843 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.005811    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.006034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.006014    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.006209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.006193    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.006390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.006375    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.006585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.006568    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.020552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.007045    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.020950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.020918    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.116519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.108616    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:00.122458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.122380    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.122777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.122749    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.127566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.127537    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrvs9\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.127791 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.127775    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.127941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.127923    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.128072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.128057    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.140438 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.140405    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.142604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.142577    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.148769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.148528    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.148894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.148764    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:00.153157 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.149642    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.153157 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.149950    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.167388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.167280    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.171781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.171719    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.172290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.172256    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.189959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.189789    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:00.230244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.206728    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:35:00.231996 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.231307    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:00.273518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.264358    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Nov 01 22:35:00.273518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.264421    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Nov 01 22:35:00.308150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.275099    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:35:00.364508 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.363057    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.364508 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.363813    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrvs9\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.388843 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.375024    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.430571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.408315    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo\") pod \"e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") " pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.434516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.416681    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:35:00.492517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.491576    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:35:00.492517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.427649    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:35:00.492517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.427597    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:00.516520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.427671    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:35:00.516520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.498967    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:35:00.516520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.499013    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:35:00.643647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.643598    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:35:00.662553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.646146    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:00.680526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.661914    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:35:00.680526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.680375    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:35:00.680526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.664116    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:35:00.680526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.680432    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:35:00.686771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.686694    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:00.814810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.814778    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:35:00.827997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.827953    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:35:00.863643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.847879    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:00.877629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.876752    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:00.932517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.928522    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:35:00.946316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:00.945594    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.141536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.126130    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:01.297780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.275669    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.401533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.400779    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body=
Nov 01 22:35:01.401533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.400838    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused"
Nov 01 22:35:01.662573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.461976    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:35:01.662573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.462035    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:35:01.662573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.515675    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.662573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.519961    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.662573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.520254    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.692542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.679919    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.805725 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.752640    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Nov 01 22:35:01.806090 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.806039    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Nov 01 22:35:01.806247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.752686    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:35:01.806374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.806358    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:35:01.861018 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.860970    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.903367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.900234    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:35:01.903367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.901813    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:35:01.903863 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.903834    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"]
Nov 01 22:35:01.911287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.911255    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:35:01.918636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.918556    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:35:01.928835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.928802    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:01.991449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.974755    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:01.991449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.974845    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:01.991449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:01.974901    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:02.139185 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.137105    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:35:02.269942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.269909    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:02.270205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.270175    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:02.317524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.291795    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:02.427731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.425682    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:35:02.587597 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.549632    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:35:02.617562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.614536    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:02.618921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.618883    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:35:02.657873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.657830    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:35:02.658184 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.658137    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:35:02.679271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.678552    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body=
Nov 01 22:35:02.679271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.678607    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused"
Nov 01 22:35:02.778144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.778107    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" event={"ID":"7f545ced-4074-41f3-be68-590aec5756e2","Type":"ContainerStarted","Data":"77c1ce074464dfac61f054adb1a09516a78022bb962caceed5da4aa9947f7a8c"}
Nov 01 22:35:02.780017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.779932    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:02.780333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.780308    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:02.780563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.780534    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:02.831384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.831348    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Nov 01 22:35:02.831648 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.831609    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Nov 01 22:35:02.844773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.844700    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:02.883543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.873658    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:02.883543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.874838    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:02.944720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:02.944671    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.064486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.064435    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"]
Nov 01 22:35:03.159715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.159614    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.281829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.277167    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:35:03.281829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.280830    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:03.281829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.280885    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.298525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.292875    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:35:03.308760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.307003    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:35:03.463771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.441846    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.588433 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.552488    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.621625 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.621580    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.653516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.651122    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.670516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.654202    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.890657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.890559    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.920517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.912955    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:35:03.920517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.914341    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:03.920517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.914382    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.933223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.933200    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:03.934018 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.933998    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:35:03.935028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.935011    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:35:03.948569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.947597    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:03.981522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:03.980181    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:35:04.251564 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.251534    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:35:04.263557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.262290    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:35:04.263557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.263025    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:35:04.273970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.273937    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:04.337517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.333882    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": context deadline exceeded"
Nov 01 22:35:04.367876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.367849    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:35:04.406700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.403788    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:04.438379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.438343    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:35:04.438572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.438417    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" event={"ID":"7f545ced-4074-41f3-be68-590aec5756e2","Type":"ContainerStarted","Data":"4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7"}
Nov 01 22:35:04.464883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.464778    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:04.537507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.537458    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:35:04.577535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.573515    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:35:04.577535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.576239    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:04.582158 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.582129    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" event={"ID":"d4257a9b-63b8-4ca3-896a-080c7fb3853f","Type":"ContainerStarted","Data":"67e3181d0851103446172e15468a7d5e230db3b2773909d3df97eecf18a42567"}
Nov 01 22:35:04.761542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.695375    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:04.761542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.696871    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:04.785984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.785935    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:04.786255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.786226    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:04.813841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.812418    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:04.813841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.813459    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:35:04.860002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.859965    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:04.860228 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.860205    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:04.917117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.917074    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:04.932350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.932311    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:35:04.939944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.939913    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:35:04.952757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:04.950158    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:35:05.054737 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.054651    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:05.150078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.150049    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:05.159183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.159153    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:35:05.234514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.232783    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:35:05.237802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.237769    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:05.238045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.238019    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:05.374589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.342689    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podStartSLOduration=6.342666294 podStartE2EDuration="6.342666294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:35:05.193418973 +0000 UTC m=+761.935114627" watchObservedRunningTime="2026-11-01 22:35:05.342666294 +0000 UTC m=+762.084361913"
Nov 01 22:35:05.404548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.399025    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:35:05.580266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.580235    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:35:05.580587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.580568    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:35:05.613997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.613957    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" event={"ID":"d4257a9b-63b8-4ca3-896a-080c7fb3853f","Type":"ContainerStarted","Data":"4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0"}
Nov 01 22:35:05.697764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.697671    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:05.698114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.698077    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:05.698310 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.698286    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:05.784952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.784896    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podStartSLOduration=6.771106695 podStartE2EDuration="6.771106695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:34:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:35:05.751702257 +0000 UTC m=+762.493397876" watchObservedRunningTime="2026-11-01 22:35:05.771106695 +0000 UTC m=+762.512802315"
Nov 01 22:35:05.966301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:05.966267    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:06.231978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:06.231883    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:06.232233 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:06.232201    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:06.696357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:06.696266    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:06.696357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:06.696324    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:07.199136 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.199010    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:07.239552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.236805    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:07.239552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.236885    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:07.613809 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.613773    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:35:07.707934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.707900    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:07.708314 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.708197    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:07.805207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.805170    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:35:07.929572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:07.925549    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:08.235303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:08.234843    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:08.235303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:08.234912    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:08.696128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:08.696041    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:08.696287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:08.696112    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:08.903999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:08.903259    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:09.237410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:09.236884    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:09.237410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:09.236946    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:09.698186 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:09.698081    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:09.698557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:09.698519    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:09.801520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:09.798042    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:10.043886 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:35:10.043847    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:35:10.236010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:10.235974    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:10.236294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:10.236247    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:10.696546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:10.696512    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:10.696906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:10.696566    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:10.948577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:10.947851    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:11.232518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:11.231858    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body=
Nov 01 22:35:11.232518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:11.231920    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused"
Nov 01 22:35:11.657574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:11.656930    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:35:11.696544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:11.696487    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:35:11.697015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:11.696579    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:35:12.252062 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:12.252028    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:35:12.253344 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:12.253312    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:35:12.265164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:12.262789    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:35:12.328023 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:12.327991    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:12.674389 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:12.674311    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:13.708520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:13.706734    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:13.708520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:13.706842    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:13.790814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:13.790780    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:13.792207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:13.792061    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:13.798749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:13.797692    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:35:13.974566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:13.966183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:14.069196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.069169    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"]
Nov 01 22:35:14.103525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.093652    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.106224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.106186    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"]
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151072    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151139    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151205    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151330    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151370    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151487    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9td5\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151560    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151623    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.157729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.151652    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292434    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292514    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292582    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292709    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292744    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292864    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9td5\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292921    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.292986    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.297657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.293016    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.319042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.311420    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.322517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.320513    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.322517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.320777    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.322517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.321244    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.349275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.349241    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.372844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.350617    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.382811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.382776    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9td5\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.424516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.384004    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.424516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.384252    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.566211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.566121    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:14.589518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.589472    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"]
Nov 01 22:35:14.642859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.623804    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:14.642859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.626159    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:35:14.764810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.764775    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"]
Nov 01 22:35:14.791829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.791698    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.812061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.810938    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:14.929841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.929804    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.956969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.956921    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.957385 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.957363    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.957770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.957740    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.975404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.973661    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25kxf\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.975404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.973801    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.975404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.973941    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.975404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.974111    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:14.975404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:14.974275    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.075830    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.078720    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.078789    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.078989    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.079019    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.079075    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.079163    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.079197    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25kxf\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.079223    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.079259    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.082212    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.082594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.082486    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.083375 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.082783    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.152292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.152249    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.160519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.154538    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.250567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.193607    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.250567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.199600    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.250567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.220168    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.254525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.251890    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.367867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.367783    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.395068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.394994    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.416515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.412013    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.431144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.422869    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:35:15.547522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.533280    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25kxf\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf\") pod \"e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") " pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.602113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.600064    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.602113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.600786    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:15.624663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.624452    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.627529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.625295    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:35:15.627529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.626345    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:15.627529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.626633    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:35:15.814100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.814056    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:35:15.815567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:15.815541    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:16.076522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.065776    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"]
Nov 01 22:35:16.116887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.086640    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:16.116887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.087355    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:16.234655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.205599    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:16.234655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.232647    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:16.234655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.233365    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:16.236009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.235094    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:16.275243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.236350    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:35:16.320010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.319963    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:16.320273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.320242    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:16.320455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.320434    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:16.346622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.339134    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:16.406581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.406161    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:16.407275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.407249    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:16.480872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.480753    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:35:16.480872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.480815    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:35:16.490835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.490325    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:16.536121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.513388    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:35:16.536121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.533161    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" event={"ID":"3324062a-c889-4668-8c46-29ab982b259c","Type":"ContainerStarted","Data":"5964d058104aca6e03c82d631c970167fc0e727f4d351363d39252b03022005c"}
Nov 01 22:35:16.573682 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.553197    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:16.573682 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.564940    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:16.632978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.632531    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:16.884584 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:35:16.884549    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38efc380_1a2f_4b7d_b2c5_f2aa86810e6c.slice/crio-9b894ca0bba9057c01d33c0a0647fba610a85dfa68eba2fea4f33894f7f10020 WatchSource:0}: Error finding container 9b894ca0bba9057c01d33c0a0647fba610a85dfa68eba2fea4f33894f7f10020: Status 404 returned error can't find the container with id 9b894ca0bba9057c01d33c0a0647fba610a85dfa68eba2fea4f33894f7f10020
Nov 01 22:35:16.885439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.885399    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"]
Nov 01 22:35:16.887132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.887097    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:16.887251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.887177    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:16.887251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:16.887226    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:17.564575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:17.563790    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" event={"ID":"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c","Type":"ContainerStarted","Data":"1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e"}
Nov 01 22:35:17.564575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:17.563861    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" event={"ID":"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c","Type":"ContainerStarted","Data":"9b894ca0bba9057c01d33c0a0647fba610a85dfa68eba2fea4f33894f7f10020"}
Nov 01 22:35:17.576374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:17.576339    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" event={"ID":"3324062a-c889-4668-8c46-29ab982b259c","Type":"ContainerStarted","Data":"32102371113d6862e7e257b4a86c2c1f90c6e744413df740dab2114bb682c297"}
Nov 01 22:35:17.594518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:17.588736    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podStartSLOduration=3.58871818 podStartE2EDuration="3.58871818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:35:17.586927244 +0000 UTC m=+774.328622865" watchObservedRunningTime="2026-11-01 22:35:17.58871818 +0000 UTC m=+774.330413801"
Nov 01 22:35:17.602177 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:17.602129    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:17.615652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:17.615603    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podStartSLOduration=4.615590564 podStartE2EDuration="4.615590564s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:35:17.612736709 +0000 UTC m=+774.354432341" watchObservedRunningTime="2026-11-01 22:35:17.615590564 +0000 UTC m=+774.357286183"
Nov 01 22:35:18.065519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:18.061859    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:18.576521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:18.575890    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:18.576521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:18.576039    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 01 22:35:18.576521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:18.576082    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 01 22:35:18.604320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:18.604289    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 01 22:35:18.604486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:18.604332    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 01 22:35:19.145530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:19.142328    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:19.579745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:19.577741    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 01 22:35:19.579745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:19.577805    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 01 22:35:19.602545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:19.602507    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 01 22:35:19.602753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:19.602554    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 01 22:35:20.570405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:20.569924    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 01 22:35:20.570405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:20.570008    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 01 22:35:20.601562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:20.601524    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:20.602148 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:20.602120    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 01 22:35:20.602426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:20.602398    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 01 22:35:21.590230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:21.588081    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:21.602876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:21.595566    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 01 22:35:21.602876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:21.595621    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 01 22:35:21.608043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:21.607814    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 01 22:35:21.608043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:21.607880    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 01 22:35:22.572962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:22.572923    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 01 22:35:22.573371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:22.573303    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 01 22:35:22.611754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:22.605761    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 01 22:35:22.611754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:22.605838    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 01 22:35:22.689032 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:22.688994    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:23.567194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:23.567158    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused" start-of-body=
Nov 01 22:35:23.567391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:23.567222    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": dial tcp 10.132.0.80:15021: connect: connection refused"
Nov 01 22:35:23.602896 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:23.602858    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Nov 01 22:35:23.603082 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:23.602932    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Nov 01 22:35:23.776440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:23.776399    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:24.586000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.585964    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:24.587170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.587143    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:24.587968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.587946    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:35:24.874279 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.873937    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:24.882991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.882960    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:24.885005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.884979    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:24.886469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:24.886445    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:35:25.682601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:25.682564    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:26.489092 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:26.489012    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:27.362075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:27.362035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:28.233361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:28.233324    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:29.081655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.081347    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:29.687065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.687026    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"]
Nov 01 22:35:29.692414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.692388    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.707727 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.707700    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"]
Nov 01 22:35:29.770161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770123    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770200    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770274    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770361    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770399    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770430    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770533    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770583    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.770666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.770613    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb9rh\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.872594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872549    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.872765 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872659    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.872765 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872718    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.872902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872782    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.872902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872833    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.872902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872873    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872922    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.872997    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.873053    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb9rh\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.873221    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.873263    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.873732    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.873756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.873746    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.874308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.874282    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.875238 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.875213    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.875854 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.875833    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.882714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.882691    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:29.883087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:29.883068    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb9rh\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:30.006179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.006144    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:30.161139 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.158092    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:30.307077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.307044    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"]
Nov 01 22:35:30.307696 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:35:30.307670    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeaab9d5_3ace_4461_82a1_07e5b0437dac.slice/crio-3b8615106b0984ad2699cafc76c8f3314d5d191e00a0aaa2d3209043fa0c38e9 WatchSource:0}: Error finding container 3b8615106b0984ad2699cafc76c8f3314d5d191e00a0aaa2d3209043fa0c38e9: Status 404 returned error can't find the container with id 3b8615106b0984ad2699cafc76c8f3314d5d191e00a0aaa2d3209043fa0c38e9
Nov 01 22:35:30.310038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.310002    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:30.310133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.310069    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:30.310133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.310098    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:30.762614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.762568    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" event={"ID":"eeaab9d5-3ace-4461-82a1-07e5b0437dac","Type":"ContainerStarted","Data":"cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd"}
Nov 01 22:35:30.763029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.762668    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" event={"ID":"eeaab9d5-3ace-4461-82a1-07e5b0437dac","Type":"ContainerStarted","Data":"3b8615106b0984ad2699cafc76c8f3314d5d191e00a0aaa2d3209043fa0c38e9"}
Nov 01 22:35:30.796211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:30.796160    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podStartSLOduration=1.796145147 podStartE2EDuration="1.796145147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:35:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:35:30.792953202 +0000 UTC m=+787.534648821" watchObservedRunningTime="2026-11-01 22:35:30.796145147 +0000 UTC m=+787.537840765"
Nov 01 22:35:31.009519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:31.006484    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:31.013747 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:31.013665    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:31.036516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:31.031159    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:31.036516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:31.031344    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:31.946583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:31.946473    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:32.007567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:32.007121    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:32.007567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:32.007183    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:32.080518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:32.080458    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:33.006950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:33.006914    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:33.007351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:33.006981    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:33.168065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:33.168030    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:34.008511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:34.008401    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:34.008511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:34.008458    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:35.009022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.008974    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:35.022751 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.022693    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:35.193637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.193586    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.355566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.355191    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:35.418514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.418111    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.484518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.483587    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.512520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.510916    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.556947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.556639    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.558053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.558007    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.582718 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.582629    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.583617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.583588    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:35.753256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:35.753208    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": context deadline exceeded"
Nov 01 22:35:36.008942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.008854    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:36.009198 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.009167    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:36.154978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.154922    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:36.340248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.340149    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:36.421970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.421931    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:36.422887 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.422859    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:35:36.488629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.488580    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:36.489245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.489206    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:35:36.489995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.489972    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:35:36.587518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.559725    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:36.587518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.560328    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:36.639567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.587939    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:36.639567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.589477    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:35:36.789067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.764931    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:35:36.789067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.765477    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:36.814676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:36.812043    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:35:37.101182 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.098633    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:37.101182 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.098693    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:37.175515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.169244    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:35:37.175515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.170729    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:35:37.200513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.199669    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.360603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.346133    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.360603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.346808    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:37.476089 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.476045    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.503358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.503318    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.508590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.505079    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.508590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.505266    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:35:37.508590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.505412    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:35:37.609883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.607085    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.609883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.607311    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:35:37.780411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.780366    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:37.872964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.872922    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:37.873277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:37.873243    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:38.099773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.099678    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:38.100058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.100024    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:38.172635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.172583    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:38.222728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.222682    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.253644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.253609    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:35:38.253915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.253897    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:38.254086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.254074    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.541775    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.542210    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.542546    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.542825    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.543074    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.543358    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:35:38.545516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.543417    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:38.652646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.652592    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.660518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.656574    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.690517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.665552    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 01 22:35:38.797658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.691673    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:38.797658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.703588    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:38.797658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.784001    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:38.797658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.781272    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:35:38.889516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:38.889030    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:35:39.010038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.009996    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:39.013518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.010296    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:39.073531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.073407    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:39.193523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.183669    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:39.228670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.228614    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:35:39.814517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.784742    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:35:39.884570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.818839    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:35:39.894993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.894940    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:39.929519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.892726    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:39.986273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.986227    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:39.987231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.987198    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:35:39.988086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:39.988061    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:35:40.033371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.033331    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:35:40.047480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.039850    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:40.178412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.178316    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": context deadline exceeded"
Nov 01 22:35:40.179992 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.179965    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:40.180167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.180146    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:40.298519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.292981    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:40.455510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.451399    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:35:40.455510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.452476    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:40.459690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.459658    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:35:40.665513 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:35:40.657309    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:35:40.809693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.795747    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:40.893557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.884882    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:40.895074 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:35:40.895039    5057 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.433s"
Nov 01 22:35:40.895527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.895507    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:35:40.912312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.912286    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:40.920633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.895279    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:40.932625 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:40.932584    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:41.079865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.069578    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.079865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.070147    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.079865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.071800    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.079865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.071996    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.079865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.072161    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.291515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.288426    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:41.356660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.356564    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:41.395451 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.391762    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:35:41.454670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.453620    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:41.454670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.453898    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.477565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.477215    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:41.477565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.477277    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:41.503196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.503162    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:35:41.508042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.506230    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:35:41.646562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.644101    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:35:41.813869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.813836    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:35:41.840804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.812621    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:41.845590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.841460    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:35:41.848608 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.848586    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:35:41.853862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.853836    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:35:41.872513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.863674    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:35:41.881513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.878742    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:35:41.892178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.892156    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:35:41.902556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:41.901427    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:35:42.007086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:42.007051    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body=
Nov 01 22:35:42.007282 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:42.007106    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused"
Nov 01 22:35:42.719583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:42.719548    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:43.574793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:43.574745    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:44.047666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:44.047617    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:35:44.048168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:44.047684    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:35:44.370363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:44.370278    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:44.667855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:44.667776    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:44.668969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:44.668946    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:44.670164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:44.670137    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:35:45.224260 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:45.224220    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:46.110514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:46.104476    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:46.766468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:46.766435    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:47.779194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:47.779153    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:48.561588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:48.561556    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:49.483687 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:49.483652    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:50.386411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:50.386368    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:51.071194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:51.071156    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:51.878326 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:51.878289    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:52.683011 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:52.682973    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:53.403426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:53.403376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:54.180685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:54.180653    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:54.897222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:54.897174    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:55.351779 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:55.351744    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:55.586207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:55.586168    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:55.768045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:55.768012    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:55.934515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:55.929931    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:56.510964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:56.510929    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:56.667781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:56.667747    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:57.198457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:57.198421    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:57.560294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:57.560257    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:58.588183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:58.588143    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:59.161629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.161585    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:59.216115 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.216084    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"]
Nov 01 22:35:59.249881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.249848    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"]
Nov 01 22:35:59.250254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.250175    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.314609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.314572    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:59.411569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.411529    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.411769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.411580    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.411769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.411701    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.411769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.411764    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.411931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.411896    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.412021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.412005    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.412088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.412054    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.412146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.412110    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49rll\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.412279 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.412264    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.482748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.482703    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:35:59.514533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514487    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.514713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514565    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.514713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514674    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.514713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514707    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.514837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514733    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49rll\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514921    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.514979    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.515047    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.515083    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.515114    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.515243    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.515589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.515531    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.516060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.516041    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.516403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.516384    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.517649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.517627    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.517753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.517710    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.525299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.525274    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-49rll\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.527012 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.526991    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token\") pod \"e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") " pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.570418 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.570386    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:35:59.802816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.802780    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"]
Nov 01 22:35:59.816434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.816404    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.830936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.830894    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"]
Nov 01 22:35:59.841720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.841694    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:35:59.873532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.873502    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"]
Nov 01 22:35:59.876359 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:35:59.876329    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6bb450_46b0_4101_a8c5_5ba6da4e9c39.slice/crio-1f305818298dc09f3b459a16e2ad29bccbafe4b12094041b4717d43a74bb3d6a WatchSource:0}: Error finding container 1f305818298dc09f3b459a16e2ad29bccbafe4b12094041b4717d43a74bb3d6a: Status 404 returned error can't find the container with id 1f305818298dc09f3b459a16e2ad29bccbafe4b12094041b4717d43a74bb3d6a
Nov 01 22:35:59.879792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.879743    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:59.879907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.879839    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:59.879907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.879885    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:35:59.924548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.924520    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.924664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.924614    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.924722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.924672    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8jrm\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.924897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.924871    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.924953 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.924938    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.925001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.924975    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.925111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.925095    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.925154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.925122    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:35:59.925269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:35:59.925250    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026258 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026225    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026258 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026265    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026303    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8jrm\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026414    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026461    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026521    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026591    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026641    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026747    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.026756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.026751    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.027151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.027123    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.027565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.027540    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.027661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.027594    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.028132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.028108    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.030669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.030366    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.031137 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.031114    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.035080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.035057    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.036002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.035969    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8jrm\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm\") pod \"e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") " pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.147038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.146944    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:00.367325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.367291    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:00.435611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.434877    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" event={"ID":"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39","Type":"ContainerStarted","Data":"e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f"}
Nov 01 22:36:00.435611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.434949    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" event={"ID":"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39","Type":"ContainerStarted","Data":"1f305818298dc09f3b459a16e2ad29bccbafe4b12094041b4717d43a74bb3d6a"}
Nov 01 22:36:00.468564 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:36:00.467866    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22e4429_a16f_49c7_b7e0_4b2eefc359ad.slice/crio-ae94f6fb35bbff8c91f895b447271a4368eb195f66c017edad5e4fcdb1586bd8 WatchSource:0}: Error finding container ae94f6fb35bbff8c91f895b447271a4368eb195f66c017edad5e4fcdb1586bd8: Status 404 returned error can't find the container with id ae94f6fb35bbff8c91f895b447271a4368eb195f66c017edad5e4fcdb1586bd8
Nov 01 22:36:00.480759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.478251    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podStartSLOduration=1.47822828 podStartE2EDuration="1.47822828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:36:00.46844673 +0000 UTC m=+817.210142350" watchObservedRunningTime="2026-11-01 22:36:00.47822828 +0000 UTC m=+817.219923900"
Nov 01 22:36:00.480759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.478779    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:00.480759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.478896    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:00.480759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.478953    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:00.480759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.479568    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"]
Nov 01 22:36:00.512725 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.511400    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:00.571561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.571530    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:36:00.573040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.573014    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:00.573156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:00.573073    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:01.538517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:01.532142    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" event={"ID":"c22e4429-a16f-49c7-b7e0-4b2eefc359ad","Type":"ContainerStarted","Data":"17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2"}
Nov 01 22:36:01.538517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:01.532203    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" event={"ID":"c22e4429-a16f-49c7-b7e0-4b2eefc359ad","Type":"ContainerStarted","Data":"ae94f6fb35bbff8c91f895b447271a4368eb195f66c017edad5e4fcdb1586bd8"}
Nov 01 22:36:01.682515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:01.640643    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:01.709517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:01.653006    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podStartSLOduration=2.652983846 podStartE2EDuration="2.652983846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:35:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:36:01.609913992 +0000 UTC m=+818.351609612" watchObservedRunningTime="2026-11-01 22:36:01.652983846 +0000 UTC m=+818.394679470"
Nov 01 22:36:01.709778 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:01.709741    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:01.889132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:01.888096    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:02.075820 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.075777    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.157611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.157459    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:02.177528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.174605    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.195660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.195621    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:02.195938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.195907    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:02.205667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.205624    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.280102 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.280050    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.422585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.421603    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.518585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.517590    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.555517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.554207    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.571167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.571125    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:02.571439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.571408    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:02.610517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.609597    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.792516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.788647    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.868520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.859584    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.890517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.889608    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:02.890517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:02.890262    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:03.010517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.007608    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:03.140084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.139973    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:03.251458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.159605    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:03.251458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.245433    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:03.251458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.187172    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:03.251458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.246070    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:36:03.251458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.246125    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:36:03.284631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.283794    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:03.284631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.284160    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:36:03.603926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.595007    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:03.668862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.667669    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:03.675644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.675587    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:03.675957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.675914    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:03.697062 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.684643    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:03.738555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.724970    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:03.785187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.752820    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:36:03.805521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.802118    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.037392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:03.998875    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:04.062515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.036840    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.376597 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.333849    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": context deadline exceeded"
Nov 01 22:36:04.415033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.412347    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:04.527516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.510963    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:04.845627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.527807    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:04.880561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.688944    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.881121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.689008    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.889518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.737666    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.905519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.756627    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.919912 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.756685    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:04.973545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:04.817187    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:04.977638 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:36:04.977606    5057 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.488s"
Nov 01 22:36:05.349998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.349964    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:36:05.361918 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.057104    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:05.362190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.109995    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.375683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.110040    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:05.375683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.110075    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.384679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.000603    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:05.384865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.384714    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:05.384865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.362004    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:36:05.384865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.384845    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:05.385030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.384888    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:36:05.385030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.384921    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:36:05.385030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.384955    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:36:05.385030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.384990    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:36:05.385030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.385025    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:36:05.385269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.385054    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:36:05.385269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.041047    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:05.385269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.385100    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:05.419231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.247683    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:05.419231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.391067    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.474538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.467748    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:05.474538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.467812    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:05.646566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.643047    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:05.657520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.646802    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:05.738516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.733031    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:05.788846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.661483    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.788846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.766543    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.788846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.782646    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:05.788846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.782703    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.858008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.855776    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:05.905099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.891442    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:05.905661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.903006    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.925060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.903756    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:05.925636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.903791    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.926298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.924838    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.947371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.926700    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:05.948864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.948827    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.998762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.998713    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:05.999114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.999063    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:05.999421 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:05.999396    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:06.151228 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.149400    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:06.151228 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.149454    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:06.397518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.372811    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:06.499518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.397796    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:06.499518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.397832    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:06.499518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.484409    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:36:06.583518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.579319    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:06.583518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.579394    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:06.627984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.627939    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:06.628256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.628224    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:06.859778 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.859739    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:06.924789 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.924755    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:06.926646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.926430    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:06.927284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.926481    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:06.931718 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.926523    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:06.958950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.958870    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:06.961973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:06.924689    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:07.001770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.001730    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:07.160034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.159942    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:07.160288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.160253    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:07.491292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.491248    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:07.493007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.492974    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:07.509059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.494106    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:07.517665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.506700    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:07.570824 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.570758    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:07.585630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.585589    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:07.585894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.585868    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:07.640872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.640369    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:07.640872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.640443    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:07.873643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.861631    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:07.897847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.897813    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:36:07.927584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.927540    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:07.998517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.973559    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.014844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.999618    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:08.041389 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:07.999686    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.042047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.009736    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.151313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.151225    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:08.151592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.151560    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:08.507711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.504079    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.507711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.504474    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:08.516775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.516737    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.521626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.521578    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.572632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.572582    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.573506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.573462    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:08.573712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.573687    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:08.643658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.643615    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:08.643987 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.643937    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.863736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.863624    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.929518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.928585    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:08.930667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.930168    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:08.982705 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:08.982644    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.033651 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.033602    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.034970 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.034932    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.058643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.058593    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.059249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.059213    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.151575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.150637    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:09.151575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.150723    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:09.381661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.381611    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.507705 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.507648    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:09.508567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.507928    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:09.563940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.563892    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.609284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.589631    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:09.609284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.589694    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:09.893883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.893766    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.943581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.929598    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:09.943581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.930174    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:36:09.943581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:09.931344    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:10.037645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.037590    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.057130 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.049171    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.078053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.077920    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:10.116553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.091311    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:10.117350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.117242    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": context deadline exceeded"
Nov 01 22:36:10.162687 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.117925    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:10.162687 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.118253    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:10.165173 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.165140    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.193949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.167533    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:10.194243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.194212    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:10.519523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.510178    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:10.519523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.510589    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.647532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.603148    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:10.647532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.603223    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.647532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.622252    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.647532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.622617    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:36:10.647532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.642641    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:10.647532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.642703    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:10.906156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.906065    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:10.943287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.929887    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:10.943549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:10.935823    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:11.059517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.041666    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.060950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.060911    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.205860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.180772    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.238313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.207588    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:11.246890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.246852    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:11.247158 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.247128    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:11.259540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.248623    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.557895 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:36:11.557780    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:36:11.584269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.584204    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.585845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.585713    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.628403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.628351    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:11.763528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.689072    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:11.763528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.689136    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:11.897751 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.897662    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:36:11.898086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.898068    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:36:11.946657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.946607    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:36:11.956528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.950861    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:11.969458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:11.969415    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:12.011806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.011764    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:12.044346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.043072    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:12.241533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.241477    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:12.274713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.274662    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:12.331787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.275824    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:12.331787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.292562    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:12.331787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.298193    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": context deadline exceeded"
Nov 01 22:36:12.394681 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.392769    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:36:12.578566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.571834    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:12.578566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.571894    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:12.592919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.590782    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:12.593827 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.593748    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:12.631527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.630594    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:12.744528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.742998    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:12.806999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.806737    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:36:12.902800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.902088    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:36:12.902800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.902693    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:12.917365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.917341    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:36:12.962674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:12.954694    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:13.073546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.011712    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:36:13.073546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.012547    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:13.073546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.013647    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:36:13.073546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.055436    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:13.172844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.172761    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:13.173065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.173040    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:13.213776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.186052    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:13.213776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.192286    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:36:13.213776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.193748    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:36:13.213776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.206710    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:36:13.581029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.580993    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:13.581663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.581628    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:13.732349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:13.732300    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:14.159068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:14.159034    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:14.159340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:14.159299    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:14.600870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:14.600822    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:14.601383 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:14.600892    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:14.601383 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:14.601251    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:36:14.952137 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:14.952102    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:15.159512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:15.147934    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:15.159512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:15.149118    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:15.159512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:15.149168    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:15.571032 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:15.571000    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Nov 01 22:36:15.571251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:15.571230    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Nov 01 22:36:16.157665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:16.157626    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:16.158350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:16.158226    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:16.297352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:16.297230    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:16.743318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:16.743284    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:36:16.745943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:16.745913    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:36:17.148020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.147929    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Nov 01 22:36:17.148020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.148010    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Nov 01 22:36:17.311465 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.311431    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"]
Nov 01 22:36:17.317589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.317562    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.339070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.339027    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0500c01f-serving-cert\""
Nov 01 22:36:17.354947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.354530    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"]
Nov 01 22:36:17.385738 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.385705    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:17.458816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.458779    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.459099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.459077    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.459385 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.459369    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k7vt\" (UniqueName: \"kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.459630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.459612    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.529034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.528997    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:17.561114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.561042    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.561372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.561352    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.561677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.561660    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k7vt\" (UniqueName: \"kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.562430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.562409    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.562686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.562203    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.573723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.573696    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.579168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.579141    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.617485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.617451    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k7vt\" (UniqueName: \"kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt\") pod \"maas-api-e2e-ait-0500c01f-656df8c894-rs6n2\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.657042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.656760    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:17.978701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:17.978575    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"]
Nov 01 22:36:18.042517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.029602    5057 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 22:36:18.165519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.164814    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:18.169515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.166435    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2" event={"ID":"939e3502-de98-41ea-8e7d-6d041c1cdc34","Type":"ContainerStarted","Data":"c5c3a5c267acf70d47928ef3f41aa12006792c9446f3c4a4114cdf2940d0cfc7"}
Nov 01 22:36:18.174546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.172217    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:36:18.669858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.669754    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:18.822435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.822396    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:18.963586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:18.958466    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:19.180703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:19.180658    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2" event={"ID":"939e3502-de98-41ea-8e7d-6d041c1cdc34","Type":"ContainerStarted","Data":"3f143c42f24e519446452282b8684057cea784591d9ca135ec493e0b40390d3b"}
Nov 01 22:36:19.180972 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:19.180959    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:19.200450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:19.200402    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2" podStartSLOduration=1.585282887 podStartE2EDuration="2.200390069s" totalImagesPullingTime="615.107182ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:36:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:36:19.199090816 +0000 UTC m=+835.940786454" watchObservedRunningTime="2026-11-01 22:36:19.200390069 +0000 UTC m=+835.942085688"
Nov 01 22:36:19.745143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:19.745090    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:20.766097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:20.766061    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:21.631214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:21.630116    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:22.514719 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:22.514678    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:23.302823 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:23.302712    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:24.166424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:24.166392    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:25.138221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:25.138186    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:25.307031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:25.306997    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:25.444983 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:25.444892    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:26.168001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:26.167970    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:26.215356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:26.215318    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:27.101106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:27.101050    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:27.991967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:27.991921    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:28.149938 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:28.149901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:28.965127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:28.965085    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:29.243477 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.243399    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"]
Nov 01 22:36:29.249378 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.249353    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.260558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.260134    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"]
Nov 01 22:36:29.333323 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333283    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333413    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333470    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333487    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333663    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333692    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333751    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333840    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kksh\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.333920 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.333866    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.435578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435542    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.435752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435614    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.435752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435644    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.435752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435740    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.435928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435779    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.435928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435863    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435947    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.435986    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kksh\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.436023    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.436130    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.436265    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.436405    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.436625    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.436770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.436718    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.438777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.438754    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.439183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.439161    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.444784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.444754    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kksh\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.447070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.447043    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token\") pod \"e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") " pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.610350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.610261    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:29.907837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.907803    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"]
Nov 01 22:36:29.909586 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:36:29.909557    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b1d51d_190e_49ff_af02_f30851ac3d25.slice/crio-7a2a5d3604a6a5472a0a765406aa6244fcefb114c1b61138cce7694104c14bfc WatchSource:0}: Error finding container 7a2a5d3604a6a5472a0a765406aa6244fcefb114c1b61138cce7694104c14bfc: Status 404 returned error can't find the container with id 7a2a5d3604a6a5472a0a765406aa6244fcefb114c1b61138cce7694104c14bfc
Nov 01 22:36:29.912325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.912292    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:29.912422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.912355    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:29.912422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.912382    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:29.960592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:29.956410    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:30.312848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:30.312806    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" event={"ID":"63b1d51d-190e-49ff-af02-f30851ac3d25","Type":"ContainerStarted","Data":"9655b8594b2df34471894c360e505731b235f9d5642e79794211158f6ece300e"}
Nov 01 22:36:30.313296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:30.312869    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" event={"ID":"63b1d51d-190e-49ff-af02-f30851ac3d25","Type":"ContainerStarted","Data":"7a2a5d3604a6a5472a0a765406aa6244fcefb114c1b61138cce7694104c14bfc"}
Nov 01 22:36:30.337094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:30.337039    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podStartSLOduration=1.337019206 podStartE2EDuration="1.337019206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:36:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:36:30.33692165 +0000 UTC m=+847.078617270" watchObservedRunningTime="2026-11-01 22:36:30.337019206 +0000 UTC m=+847.078714840"
Nov 01 22:36:30.610940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:30.610856    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:31.050472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:31.050433    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:31.611648 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:31.611614    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 01 22:36:31.612025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:31.611667    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 01 22:36:31.890620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:31.890532    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:32.610597 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:32.610563    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 01 22:36:32.610762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:32.610621    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 01 22:36:32.910150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:32.910032    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:33.109958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.109924    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"]
Nov 01 22:36:33.115325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.115306    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.130803 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.130776    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Nov 01 22:36:33.132678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.132650    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-c7qwl\""
Nov 01 22:36:33.132986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.132883    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Nov 01 22:36:33.133128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.132922    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Nov 01 22:36:33.146515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.145100    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"]
Nov 01 22:36:33.308082 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308051    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.308266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308104    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.308266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308187    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.308266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308215    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.308408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308331    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.308408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308357    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84jwj\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.308504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.308482    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.409995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.409962    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.410202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.410058    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.410202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.410096    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84jwj\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.410333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.410212    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.410333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.410325    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.410436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.410366    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.410436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.410402    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.411260 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.411195    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.412731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.412711    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.412978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.412956    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.413026 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.412999    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.413151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.413129    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.441165 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.441125    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.453522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.451234    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-84jwj\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj\") pod \"istiod-openshift-gateway-7ddd9cfdc8-246qm\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.471096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.471069    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:33.611811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.611313    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 01 22:36:33.611811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.611370    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 01 22:36:33.794335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.794236    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"]
Nov 01 22:36:33.795957 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:36:33.795930    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod619f00b4_a838_4c64_96e4_7e1fcb83d1fe.slice/crio-a4fe23bd61d9ab7f3260f08e86b10b116127aa94a5db5e8a5cab980b862cc6ff WatchSource:0}: Error finding container a4fe23bd61d9ab7f3260f08e86b10b116127aa94a5db5e8a5cab980b862cc6ff: Status 404 returned error can't find the container with id a4fe23bd61d9ab7f3260f08e86b10b116127aa94a5db5e8a5cab980b862cc6ff
Nov 01 22:36:33.801050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:33.801009    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:34.377437 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:34.377400    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm" event={"ID":"619f00b4-a838-4c64-96e4-7e1fcb83d1fe","Type":"ContainerStarted","Data":"a4fe23bd61d9ab7f3260f08e86b10b116127aa94a5db5e8a5cab980b862cc6ff"}
Nov 01 22:36:34.603265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:34.603223    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:34.611846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:34.611802    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 01 22:36:34.612138 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:34.612107    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 01 22:36:35.585532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:35.584688    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:35.611150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:35.611107    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 01 22:36:35.611402 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:35.611166    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 01 22:36:36.611072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:36.611040    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Nov 01 22:36:36.611549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:36.611094    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Nov 01 22:36:36.930080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:36.929999    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:37.763258 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:37.759479    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:38.693320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.584464    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Nov 01 22:36:38.693320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.684694    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:38.693320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.684732    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:38.693320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.684775    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:38.693320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.684801    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": context deadline exceeded"
Nov 01 22:36:38.693320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.692440    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:38.745513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.744636    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:38.745513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.744702    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:38.844550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.839938    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:39.004519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.972740    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:39.038247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.974113    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:36:39.038247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.015639    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:39.038247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.974162    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:39.038247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:38.974143    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:39.038247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.016481    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:39.043250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.043217    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:39.374105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.373782    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:39.692572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.689393    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:39.692572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.690216    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:36:39.849801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.849767    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:39.850424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:39.850393    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:40.077521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.040535    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:40.077521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.040924    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:40.077521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.041176    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:40.077521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.041772    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:36:40.077521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.041843    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:36:40.077521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.041876    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:36:40.094517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.081008    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:40.094517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.081567    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:36:40.132630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.132581    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": context deadline exceeded"
Nov 01 22:36:40.133936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.133866    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:40.134174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.134153    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:36:40.144652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.144611    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:40.285542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.285487    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:40.497236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.497180    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:40.711800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.711759    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:40.718759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.718668    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:40.893573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.887648    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:40.893573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.887715    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.002515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:40.975922    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.003327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.003292    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:41.005937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.005894    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.018643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.011627    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:41.018877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.018849    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.058665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.058604    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:41.075905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.075861    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.160577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.105272    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.184865 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:36:41.161115    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:36:41.266745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.181314    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.388775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.388729    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:41.414576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.389043    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.482528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.477672    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:41.633443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.633401    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:41.685566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.659554    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.722555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.695580    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:41.722555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.695650    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.724680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.723334    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.725247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.725218    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:41.903208 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.903166    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:41.983030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:41.982953    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.073559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.056608    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:42.073559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.056674    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.083516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.076288    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:42.305588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.115641    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.305588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.170711    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.308738 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.189967    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.321682 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.190048    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:42.501546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.217753    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:42.501546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.249541    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:42.501546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.270053    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.511825 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.511795    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:36:42.525662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.525624    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:36:42.756978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.756924    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.769719 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.757107    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.860634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.783828    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:42.991259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:42.991199    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.142784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.142628    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:43.146155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.142935    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.309876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.309831    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:43.341236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.341202    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:43.350556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.350424    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:43.351307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.351029    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:43.353762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.353743    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:43.377516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.368854    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.501753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.486184    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.501753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.493785    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:43.513522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.509714    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.645520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.633974    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.869304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.869201    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.869775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.869733    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:43.870135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.870110    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:43.908175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:43.908142    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:36:44.041478 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.039626    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.042507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.042094    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:36:44.136759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.136566    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": context deadline exceeded"
Nov 01 22:36:44.364531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.352210    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.414087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.413989    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:44.414462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.413641    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.489088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.489042    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.637750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.637473    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:44.638094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.527559    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.639108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.520044    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.639757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.639736    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:36:44.907629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.907582    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:44.942623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.942580    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:44.976578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:44.976538    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.040736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.040697    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:45.041024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.040991    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.133518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.132378    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.359519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.358692    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.435641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.435369    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.440518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.437258    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.613577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.613276    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.638517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.638050    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.651976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.639505    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.651976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.641254    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.651976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.642018    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.725684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.725636    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:45.725994 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.725945    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.747929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.747681    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:45.915022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.914926    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:36:45.955743 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:45.953195    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:46.004606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.002594    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded"
Nov 01 22:36:46.114753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.114714    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:46.134736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.134688    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:46.359638 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.359588    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:36:46.448517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.435631    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:46.467531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.461593    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:46.650578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.648775    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:46.650578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.649158    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:46.650578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.649425    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:46.650578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.649686    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:46.916689 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.916580    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:46.980599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:46.971607    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.012673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.012624    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.137653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.137607    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:36:47.364702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.364651    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.489518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.448425    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.563175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.502808    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": context deadline exceeded"
Nov 01 22:36:47.563395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.562605    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:47.563570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.563543    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.563693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.562663    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.653689 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.653598    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:36:47.678520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.653911    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.678828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.662755    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:47.679251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.662787    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:47.679680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.662806    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:47.701111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.662825    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:36:47.919727 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.919627    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:47.989643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:47.989592    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.017943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.017896    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.139510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.139448    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.173630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.173544    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:48.371072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.369607    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.568374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.568330    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.570128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.570089    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:48.583729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.583695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:48.618588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.618338    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/health\": context deadline exceeded"
Nov 01 22:36:48.651570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.626956    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.685298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.684609    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:36:48.686040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.686009    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:48.688613 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.688569    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:36:48.698706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.698665    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:48.701475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.701442    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.004763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:48.975214    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.034156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.007193    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": context deadline exceeded"
Nov 01 22:36:49.034156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.033568    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.172571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.158226    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.348714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.345681    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:49.371871 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.371828    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.416701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.416648    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.508458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.508423    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:49.570764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.569689    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:36:49.572449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.571622    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.662888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.662806    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:49.686441 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.686402    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:36:49.687311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.687281    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.703612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.703570    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.748372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.747613    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:49.940834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:49.940734    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:36:50.019550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.013633    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.054582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.048161    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.140519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.133537    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:36:50.142393 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.142066    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.162515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.161594    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.195077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.194872    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:50.216879 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.216851    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:36:50.223511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.221432    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:36:50.318515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.298984    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:36:50.404626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.376572    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.405283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.405252    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:50.587521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.575907    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:36:50.667472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.667309    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:36:50.764655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.764514    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.799570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.790822    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:36:50.805515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.802359    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:36:50.837224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.837159    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:36:50.863305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.861871    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:36:50.960086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.957318    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:36:50.973243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:50.972289    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:36:51.382629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:51.382596    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:51.529254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:51.528486    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:51.707568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:51.694130    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:52.469379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:52.468993    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:52.501638 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:52.501601    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:52.501762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:52.501682    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:36:53.232203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:53.232164    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm" event={"ID":"619f00b4-a838-4c64-96e4-7e1fcb83d1fe","Type":"ContainerStarted","Data":"9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8"}
Nov 01 22:36:53.267795 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:53.267447    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm" podStartSLOduration=1.565508917 podStartE2EDuration="20.267426052s" totalImagesPullingTime="18.701917135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:36:53.263706135 +0000 UTC m=+870.005401756" watchObservedRunningTime="2026-11-01 22:36:53.267426052 +0000 UTC m=+870.009121666"
Nov 01 22:36:53.297761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:53.297718    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:54.173610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:54.173570    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:54.240425 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:54.240389    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:54.242595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:54.242570    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:36:54.912952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:54.912912    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:55.736004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:55.735966    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:56.481744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:56.481703    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:56.669327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:56.669288    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:57.129621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.129580    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"]
Nov 01 22:36:57.135696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.135656    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2" podUID="939e3502-de98-41ea-8e7d-6d041c1cdc34" containerName="maas-api" containerID="cri-o://3f143c42f24e519446452282b8684057cea784591d9ca135ec493e0b40390d3b" gracePeriod=30
Nov 01 22:36:57.191039 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.190791    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:57.454563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.450364    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:57.709977 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.709933    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:57.758832 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.758471    5057 generic.go:356] "Generic (PLEG): container finished" podID="939e3502-de98-41ea-8e7d-6d041c1cdc34" containerID="3f143c42f24e519446452282b8684057cea784591d9ca135ec493e0b40390d3b" exitCode=0
Nov 01 22:36:57.758832 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.758546    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2" event={"ID":"939e3502-de98-41ea-8e7d-6d041c1cdc34","Type":"ContainerDied","Data":"3f143c42f24e519446452282b8684057cea784591d9ca135ec493e0b40390d3b"}
Nov 01 22:36:57.791985 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.791955    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:57.868284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.868212    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs\") pod \"939e3502-de98-41ea-8e7d-6d041c1cdc34\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") "
Nov 01 22:36:57.868526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.868323    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca\") pod \"939e3502-de98-41ea-8e7d-6d041c1cdc34\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") "
Nov 01 22:36:57.868526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.868385    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls\") pod \"939e3502-de98-41ea-8e7d-6d041c1cdc34\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") "
Nov 01 22:36:57.868526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.868469    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt\" (UniqueName: \"kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt\") pod \"939e3502-de98-41ea-8e7d-6d041c1cdc34\" (UID: \"939e3502-de98-41ea-8e7d-6d041c1cdc34\") "
Nov 01 22:36:57.869931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.869901    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca" pod "939e3502-de98-41ea-8e7d-6d041c1cdc34" (UID: "939e3502-de98-41ea-8e7d-6d041c1cdc34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:36:57.873348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.873321    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs" pod "939e3502-de98-41ea-8e7d-6d041c1cdc34" (UID: "939e3502-de98-41ea-8e7d-6d041c1cdc34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:36:57.875997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.875964    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt" pod "939e3502-de98-41ea-8e7d-6d041c1cdc34" (UID: "939e3502-de98-41ea-8e7d-6d041c1cdc34"). InnerVolumeSpecName "kube-api-access-7k7vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:36:57.877842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.877814    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls" pod "939e3502-de98-41ea-8e7d-6d041c1cdc34" (UID: "939e3502-de98-41ea-8e7d-6d041c1cdc34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:36:57.955050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.954934    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:57.977567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.971296    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:36:57.977567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.971327    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/939e3502-de98-41ea-8e7d-6d041c1cdc34-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:36:57.977567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.971339    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/939e3502-de98-41ea-8e7d-6d041c1cdc34-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:36:57.977567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:57.971352    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k7vt\" (UniqueName: \"kubernetes.io/projected/939e3502-de98-41ea-8e7d-6d041c1cdc34-kube-api-access-7k7vt\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:36:58.126720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.113974    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:58.604074 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.604035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:58.757347 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.757309    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:58.769697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.769662    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2" event={"ID":"939e3502-de98-41ea-8e7d-6d041c1cdc34","Type":"ContainerDied","Data":"c5c3a5c267acf70d47928ef3f41aa12006792c9446f3c4a4114cdf2940d0cfc7"}
Nov 01 22:36:58.769895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.769727    5057 scope.go:122] "RemoveContainer" containerID="3f143c42f24e519446452282b8684057cea784591d9ca135ec493e0b40390d3b"
Nov 01 22:36:58.769895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.769872    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"
Nov 01 22:36:58.802049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.802014    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"]
Nov 01 22:36:58.805307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.805278    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0500c01f-656df8c894-rs6n2"]
Nov 01 22:36:58.953270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:58.953232    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:59.093249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:59.093127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:59.277565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:59.275443    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:59.404813 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:59.404778    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="939e3502-de98-41ea-8e7d-6d041c1cdc34" path="/var/lib/kubelet/pods/939e3502-de98-41ea-8e7d-6d041c1cdc34/volumes"
Nov 01 22:36:59.431933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:59.417127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:59.555085 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:59.555005    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:36:59.734025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:36:59.733992    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:00.329633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:00.329596    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:01.353546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:01.353480    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:02.181934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:02.181897    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:02.821167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:02.821113    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:02.981319 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:02.981282    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:03.113426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:03.113340    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:03.396041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:03.395947    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:03.553526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:03.553411    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:04.650518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:04.647502    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:05.379233 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:05.379196    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:05.936599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:05.936551    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:06.087541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:06.087486    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:06.216961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:06.214037    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:06.346884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:06.344806    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:07.182091 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:07.182058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:07.734552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:07.734520    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:08.236344 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:08.236307    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:08.370222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:08.370187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:08.815785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:08.815754    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:08.979333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:08.979282    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:09.282999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:09.282945    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:09.431129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:09.430553    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:10.018549 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:37:10.018510    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:37:10.567403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:10.567357    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:11.561017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:11.560978    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:11.688515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:11.687554    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:11.815916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:11.815812    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:12.893854 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:12.893821    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:13.053466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:13.053419    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:13.200566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:13.199907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:13.343618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:13.339045    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:13.481969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:13.481870    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:37:14.079984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:14.079949    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:15.081940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:15.081900    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:15.537074 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:15.537041    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:16.684204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:16.684170    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:17.865402 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:17.865368    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:18.808329 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:18.808278    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:19.983521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:19.983464    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:20.124100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:20.124061    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:20.734692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:20.734655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:21.484155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:21.484111    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:21.612857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:21.612821    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:21.805401 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:21.805312    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:22.562113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:22.562075    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:23.305190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:23.305155    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:23.498579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:23.498553    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:37:23.499568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:23.499547    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:37:24.066225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:24.066188    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:25.173437 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:25.173390    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:26.030463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:26.030427    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:26.677847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:26.677814    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:27.358680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:27.358642    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:27.956217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:27.956179    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:28.655300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:28.655256    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:28.850515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:28.848022    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:28.982716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:28.981549    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:29.317599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:29.317310    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:29.495030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:29.494992    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:30.289976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:30.289942    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:30.449843 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:30.449719    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:30.891549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:30.891477    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:31.017096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:31.017058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:31.457318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:31.457281    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:32.354502 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:32.354465    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:32.968567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:32.968526    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:33.718355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:33.718319    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:34.593717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:34.593591    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:35.234576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:35.234539    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:35.498663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:35.498560    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:35.977400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:35.977354    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:36.380100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:36.380004    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:36.555655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:36.555614    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:36.999322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:36.999282    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:37.728281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:37.728240    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:38.339947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:38.339913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:38.462506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:38.462451    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:38.972015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:38.971980    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:39.627692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:39.627655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:40.016263 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:37:40.016226    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:37:40.303630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:40.303540    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:40.845941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:40.845907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:40.975448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:40.975411    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:41.097737 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:41.097629    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:41.301527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:41.300656    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:41.453572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:41.452995    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:42.026716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:42.026681    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:42.236867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:42.236835    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:42.582768 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:42.582728    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:42.742530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:42.741127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:43.308416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:43.308379    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:43.429030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:43.426099    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:43.567839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:43.567740    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:44.338581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:44.338546    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:44.988909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:44.988856    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:45.952901 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:45.952865    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:46.428956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:46.428850    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:46.577699 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:46.577664    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:46.851877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:46.851833    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:47.595021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:47.594990    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:47.728692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:47.728655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:47.893971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:47.893875    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:48.325339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:48.325301    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:48.447847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:48.447807    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:48.767654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:48.767615    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:49.252665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:49.252632    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:49.529859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:49.529738    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:49.963158 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:49.963120    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:50.468987 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:50.468951    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:50.705829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:50.705745    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:51.138129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:51.138096    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:51.285516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:51.281563    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:37:51.451703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:51.414691    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:51.590513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:51.590466    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:52.778091 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:52.778048    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:53.584430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:53.584376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:53.806760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:53.806724    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:54.608689 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:54.608654    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:55.178792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:55.178757    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:55.552471 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:55.552429    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:55.706890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:55.706856    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:56.047081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:56.046999    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:56.315568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:56.300791    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:56.528291 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:56.528255    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:56.678704 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:56.677151    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:57.268940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:57.268896    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:57.746742 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:57.746688    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:57.907890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:57.907851    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:58.365224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:58.365191    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:59.084384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:59.084351    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:37:59.877896 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:37:59.877848    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:00.609561 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:00.609524    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:01.280771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:01.280732    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:02.083950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:02.083913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:02.658175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:02.658139    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:02.804250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:02.804208    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:03.632518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:03.631926    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:04.205585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.205555    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"]
Nov 01 22:38:04.206232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.206217    5057 cpu_manager.go:414] "containerMap: removing container" podUID="939e3502-de98-41ea-8e7d-6d041c1cdc34" containerName="maas-api"
Nov 01 22:38:04.206279 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.206234    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="939e3502-de98-41ea-8e7d-6d041c1cdc34" containerName="maas-api"
Nov 01 22:38:04.206346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.206334    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="939e3502-de98-41ea-8e7d-6d041c1cdc34" containerName="maas-api"
Nov 01 22:38:04.210008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.209986    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.218436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.218207    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-10a1d9-serving-cert\""
Nov 01 22:38:04.229480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.229455    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"]
Nov 01 22:38:04.263278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.263242    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.263657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.263636    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.264390 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.264365    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqdzv\" (UniqueName: \"kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.264473 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.264447    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.343227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.343184    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:04.366371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.366334    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.366566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.366447    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.366623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.366557    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqdzv\" (UniqueName: \"kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.366670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.366630    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.367290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.367267    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.368960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.368940    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.369158 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.369124    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.374752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.374732    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqdzv\" (UniqueName: \"kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv\") pod \"maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.474249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.474159    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:04.532302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.532267    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:04.875549 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.873207    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"]
Nov 01 22:38:04.981248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:04.981207    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x" event={"ID":"848cd2d7-99cb-499a-b13a-19782bfed719","Type":"ContainerStarted","Data":"9b1bbbdfea11368997f889961f68f9c00b689bace4525b25dc796cd4c4a1e189"}
Nov 01 22:38:05.279885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:05.279848    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:06.075156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:06.072239    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:06.075156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:06.074095    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x" event={"ID":"848cd2d7-99cb-499a-b13a-19782bfed719","Type":"ContainerStarted","Data":"63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4"}
Nov 01 22:38:06.075156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:06.074207    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:06.440574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:06.440450    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:06.901573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:06.901540    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:07.048420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:07.048383    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:07.332093 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:07.332032    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:07.497524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:07.484470    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:07.900514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:07.900214    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:08.698829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:08.698792    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:09.581750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:09.578946    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:10.017032 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:38:10.016996    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:38:10.356859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:10.356762    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:11.145956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:11.145909    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:11.412403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:11.412313    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:12.094624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:12.094592    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:38:12.113847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:12.113791    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x" podStartSLOduration=7.692818674 podStartE2EDuration="8.113777944s" totalImagesPullingTime="420.95927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:38:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:38:06.105928637 +0000 UTC m=+942.847624280" watchObservedRunningTime="2026-11-01 22:38:12.113777944 +0000 UTC m=+948.855473560"
Nov 01 22:38:12.391657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:12.391561    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:12.532211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:12.532173    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:12.964410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:12.964376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:13.379102 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:13.379005    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:14.155738 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:14.155702    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:14.810593 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:14.810551    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:15.630469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:15.630428    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:16.259441 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:16.259366    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:16.378076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:16.378037    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:16.660729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:16.660621    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:16.941450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:16.941354    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:17.182677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:17.182638    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:17.341355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:17.341322    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:17.996046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:17.996014    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:18.660323 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:18.660271    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:19.415109 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.415069    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"]
Nov 01 22:38:19.438021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.437990    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.439379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.437889    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"]
Nov 01 22:38:19.442877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.442859    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-fa8cda-serving-cert\""
Nov 01 22:38:19.492808 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.492778    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:19.590229 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.589860    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.590229 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.589940    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf7kd\" (UniqueName: \"kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.590229 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.590017    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.590229 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.590040    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.605743 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.605705    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:19.691324 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.691237    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.691324 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.691312    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf7kd\" (UniqueName: \"kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.691617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.691358    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.691617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.691388    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.692332 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.692299    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.694541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.694483    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.696278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.696252    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.699936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.699912    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf7kd\" (UniqueName: \"kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd\") pod \"maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:19.727301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.727243    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:19.764244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:19.764212    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:20.066394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:20.066364    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"]
Nov 01 22:38:20.068128 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:38:20.067943    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70062ef4_2ebb_4b14_950a_fbadd78dbdc9.slice/crio-fa15ae8367a99f59975aa433f2a71e1dc40d7d534d58c4a7407fc2786f56a62b WatchSource:0}: Error finding container fa15ae8367a99f59975aa433f2a71e1dc40d7d534d58c4a7407fc2786f56a62b: Status 404 returned error can't find the container with id fa15ae8367a99f59975aa433f2a71e1dc40d7d534d58c4a7407fc2786f56a62b
Nov 01 22:38:20.245514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:20.245460    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc" event={"ID":"70062ef4-2ebb-4b14-950a-fbadd78dbdc9","Type":"ContainerStarted","Data":"fa15ae8367a99f59975aa433f2a71e1dc40d7d534d58c4a7407fc2786f56a62b"}
Nov 01 22:38:20.592420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:20.592382    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:21.134152 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:21.134114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:21.256379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:21.256333    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc" event={"ID":"70062ef4-2ebb-4b14-950a-fbadd78dbdc9","Type":"ContainerStarted","Data":"d839e50e1042d118dfdd89d85e05c19a275c3fe5d0078bd6b32b41176208343c"}
Nov 01 22:38:21.256619 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:21.256405    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:21.278686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:21.278633    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc" podStartSLOduration=1.904055636 podStartE2EDuration="2.278619209s" totalImagesPullingTime="374.563573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:38:21.276272426 +0000 UTC m=+958.017968044" watchObservedRunningTime="2026-11-01 22:38:21.278619209 +0000 UTC m=+958.020314826"
Nov 01 22:38:21.279712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:21.279677    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:21.617537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:21.617459    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:22.400590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:22.400554    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:22.549946 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:22.549905    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:22.690844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:22.690758    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:23.588463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:23.588431    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:24.378860 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:24.378820    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:25.100692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:25.100655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:25.897695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:25.897660    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:26.797778 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:26.797726    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:27.285834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:27.285806    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:38:27.869246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:27.869189    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:28.689199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:28.689159    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:29.391665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:29.391627    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:30.087514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:30.087132    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:30.957546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:30.956297    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"]
Nov 01 22:38:30.977587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:30.975913    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:30.977587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:30.976281    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"]
Nov 01 22:38:30.982973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:30.980209    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-41b8f5-serving-cert\""
Nov 01 22:38:31.044404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.044371    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:31.076614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.076570    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ch9xr\" (UniqueName: \"kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.077020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.077002    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.077334 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.077317    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.077528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.077513    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.188556 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.188007    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:31.211686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.211589    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.212183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.212151    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ch9xr\" (UniqueName: \"kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.213281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.213256    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.213643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.213623    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.214000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.213978    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.215298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.215274    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.216563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.216546    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.221071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.221049    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ch9xr\" (UniqueName: \"kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr\") pod \"maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.310112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.310078    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:31.316395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.316369    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:31.632669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:31.632611    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"]
Nov 01 22:38:32.282884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.282845    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:32.402952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.402896    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4" event={"ID":"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e","Type":"ContainerStarted","Data":"0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6"}
Nov 01 22:38:32.403172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.402984    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4" event={"ID":"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e","Type":"ContainerStarted","Data":"e0242e4f0851bf535b52abcd2a96f9676b75c52e21409e4efac5b6c9ef5e0def"}
Nov 01 22:38:32.403172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.403099    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:32.425975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.425915    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4" podStartSLOduration=1.876120198 podStartE2EDuration="2.425899468s" totalImagesPullingTime="549.77927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:38:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:38:32.421090595 +0000 UTC m=+969.162786215" watchObservedRunningTime="2026-11-01 22:38:32.425899468 +0000 UTC m=+969.167595088"
Nov 01 22:38:32.836409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.836369    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:32.965576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:32.965543    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:33.128627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:33.128299    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:33.293971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:33.293911    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:33.882439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:33.882401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:34.506234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:34.506197    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:35.364627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:35.364587    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:36.092526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:36.091047    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:36.235577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:36.235535    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:36.793554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:36.793512    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:37.581263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:37.581227    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:38.033721 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.033685    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:38.289403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.289318    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:38.415606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.415525    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:38:38.444810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.444777    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:38.690487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.690399    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"]
Nov 01 22:38:38.696763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.696725    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.728487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.723376    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-425870c1-openshift-default-dockercfg-lrgxq\""
Nov 01 22:38:38.729913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.729886    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"]
Nov 01 22:38:38.805821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.804323    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:38.894514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.893689    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937239    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937438    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937583    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937609    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937738    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937785    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937824    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrlmd\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:38.939516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:38.937894    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.017969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.017925    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.046817    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.046943    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.046985    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047095    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047136    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047169    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrlmd\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047225    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047281    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047342    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047722    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.047731    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.048032    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.048283    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.051514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.048406    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.057183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.057150    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.095511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.087539    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.178578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.176149    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.178578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.177332    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrlmd\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd\") pod \"e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") " pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.318204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.318084    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:39.415105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.415068    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:39.658467 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:38:39.655461    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf842a24a_6ba4_46b9_a3e4_7a5c624a27bf.slice/crio-3cfb60cd12878393b4d2ca2bd86b353d198ab78659a7778a5ffed17a78909af1 WatchSource:0}: Error finding container 3cfb60cd12878393b4d2ca2bd86b353d198ab78659a7778a5ffed17a78909af1: Status 404 returned error can't find the container with id 3cfb60cd12878393b4d2ca2bd86b353d198ab78659a7778a5ffed17a78909af1
Nov 01 22:38:39.663113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.663081    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"]
Nov 01 22:38:39.667617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.667572    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:38:39.667743 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.667670    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:38:39.667743 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:39.667714    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:38:40.052004 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:38:40.049696    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:38:40.066530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.065704    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:40.185453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.185420    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:40.328464 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.328380    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:40.496558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.496522    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" event={"ID":"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf","Type":"ContainerStarted","Data":"c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"}
Nov 01 22:38:40.496735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.496579    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" event={"ID":"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf","Type":"ContainerStarted","Data":"3cfb60cd12878393b4d2ca2bd86b353d198ab78659a7778a5ffed17a78909af1"}
Nov 01 22:38:40.521711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.521659    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podStartSLOduration=2.52164404 podStartE2EDuration="2.52164404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:38:40.51796678 +0000 UTC m=+977.259662421" watchObservedRunningTime="2026-11-01 22:38:40.52164404 +0000 UTC m=+977.263339657"
Nov 01 22:38:40.999400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:40.999363    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:41.127100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:41.127058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:41.258506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:41.258410    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:41.319885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:41.319849    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:41.329040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:41.328992    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:41.504080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:41.504031    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:41.505093 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:41.505072    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:38:42.170915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:42.170880    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:42.311755 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:42.311717    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:42.455514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:42.454922    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:42.898512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:42.898412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:43.054043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:43.054011    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:43.592379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:43.592336    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:44.084652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:44.084617    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:44.236836 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:44.224787    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:44.609420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:44.609389    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:45.649132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:45.649006    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:46.526972 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:46.526913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:46.655169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:46.655134    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:47.296384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:47.296341    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:48.074889 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:48.074855    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:48.865538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:48.865472    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:49.308222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:49.308189    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:38:49.875427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:49.875385    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:50.850259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:50.850221    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:52.016237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:52.016182    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:52.764175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:52.764119    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:53.601327 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:53.601275    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:54.009090 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:54.009045    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:54.404458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:54.404367    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:54.530363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:54.530315    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:54.653387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:54.653352    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:55.315040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:55.314916    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:55.946412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:55.946377    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:56.607047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:56.607013    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:57.443470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:57.443428    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:58.199737 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:58.199697    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:58.925668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:58.925630    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:38:59.673918 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:38:59.673873    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:00.567161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:00.567112    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:00.707267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:00.707224    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:01.252715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:01.252673    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:01.984911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:01.984861    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:02.690358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:02.690321    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:03.366129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:03.366093    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:03.991582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:03.991546    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:04.668388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:04.668356    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:05.692798 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:05.692752    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:06.349478 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:06.349433    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:06.984052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:06.984016    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:07.662236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:07.662204    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:08.030369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:08.030330    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:08.388903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:08.388804    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:09.182219 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.182182    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:09.572878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.572843    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"]
Nov 01 22:39:09.578607 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.578577    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.596815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.596789    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.596962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.596929    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.597104 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.597075    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.597230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.597162    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.597476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.597272    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.597476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.597316    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bptt\" (UniqueName: \"kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.601760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.601735    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"]
Nov 01 22:39:09.602912 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.602888    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-425870c1\"/\"default-dockercfg-56gdx\""
Nov 01 22:39:09.603070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.603018    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-425870c1\"/\"e2e-lifecycle-model-425870c1-kserve-self-signed-certs\""
Nov 01 22:39:09.660993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.660942    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:09.667782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.667745    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-425870c1\"/\"openshift-service-ca.crt\""
Nov 01 22:39:09.668295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.667780    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-425870c1\"/\"kube-root-ca.crt\""
Nov 01 22:39:09.699153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699118    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699200    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699303    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699344    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bptt\" (UniqueName: \"kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699474    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699598    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699850    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.699986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.699943    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.700247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.700223    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.702281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.702208    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.703105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.703083    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.712676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.712630    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bptt\" (UniqueName: \"kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt\") pod \"e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") " pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:09.912624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:09.911467    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:10.156799 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:39:10.093182    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:39:10.156799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.109296    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:10.305330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.305294    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"]
Nov 01 22:39:10.306313 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:39:10.306279    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25c2a203_49a6_4361_bec4_8fb7f3357c33.slice/crio-bc4e483f3faab9c75d2cbaac1826eec3da8017eb30d9908e4b144e8f14105744 WatchSource:0}: Error finding container bc4e483f3faab9c75d2cbaac1826eec3da8017eb30d9908e4b144e8f14105744: Status 404 returned error can't find the container with id bc4e483f3faab9c75d2cbaac1826eec3da8017eb30d9908e4b144e8f14105744
Nov 01 22:39:10.868919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.868879    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" event={"ID":"25c2a203-49a6-4361-bec4-8fb7f3357c33","Type":"ContainerStarted","Data":"a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f"}
Nov 01 22:39:10.868919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.868926    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" event={"ID":"25c2a203-49a6-4361-bec4-8fb7f3357c33","Type":"ContainerStarted","Data":"bc4e483f3faab9c75d2cbaac1826eec3da8017eb30d9908e4b144e8f14105744"}
Nov 01 22:39:10.893303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.893249    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" podStartSLOduration=1.89323359 podStartE2EDuration="1.89323359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:39:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:39:10.887138226 +0000 UTC m=+1007.628833845" watchObservedRunningTime="2026-11-01 22:39:10.89323359 +0000 UTC m=+1007.634929207"
Nov 01 22:39:10.898143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.898106    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:10.913620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:10.913011    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:11.589378 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:11.589338    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:12.370119 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:12.369455    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:13.164257 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:13.164204    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:13.616517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:13.600343    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:14.914562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:14.913412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:16.082518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:16.074626    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:17.034571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.033607    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:39:17.359577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.356338    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:39:17.483067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.483023    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:39:17.506078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.505604    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:39:17.506078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.505669    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:39:17.517754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.517638    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-tpppl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:39:17.517754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.517696    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:39:17.655971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.654577    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:39:17.907572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:17.907460    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:18.090514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:18.083565    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": context deadline exceeded"
Nov 01 22:39:18.279583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:18.277582    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:39:19.074753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:19.074714    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:19.912701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:19.912662    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:19.938788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:19.938756    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:20.015830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:20.015794    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:20.089994 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:20.089971    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:39:20.701684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:20.701650    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:21.434403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:21.434309    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:22.180632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:22.180595    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:22.809941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:22.809888    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:23.548854 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:23.548820    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:24.121434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.121395    5057 generic.go:356] "Generic (PLEG): container finished" podID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerID="c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0" exitCode=0
Nov 01 22:39:24.121928 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.121469    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" event={"ID":"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf","Type":"ContainerDied","Data":"c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"}
Nov 01 22:39:24.122101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.122081    5057 scope.go:122] "RemoveContainer" containerID="c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"
Nov 01 22:39:24.124364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.124325    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:39:24.124474 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.124423    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:39:24.124588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.124471    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:39:24.374125 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:24.374015    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:25.157558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:25.157519    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" event={"ID":"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf","Type":"ContainerStarted","Data":"fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145"}
Nov 01 22:39:25.195003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:25.194947    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:25.332664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:25.325841    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:39:25.337654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:25.337617    5057 patch_prober.go:29] interesting pod/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 22:39:25.337903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:25.337876    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 22:39:26.097272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:26.097229    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:26.322527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:26.321853    5057 patch_prober.go:29] interesting pod/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 22:39:26.322527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:26.321909    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 22:39:26.504784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:26.504756    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:39:26.816522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:26.816418    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:27.347516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:27.346620    5057 patch_prober.go:29] interesting pod/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 22:39:27.347516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:27.346683    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 22:39:27.853151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:27.853116    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:28.324878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:28.324840    5057 patch_prober.go:29] interesting pod/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 22:39:28.325605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:28.325568    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 22:39:28.703797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:28.703743    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:29.331012 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:29.330948    5057 patch_prober.go:29] interesting pod/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 22:39:29.331301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:29.331271    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 22:39:29.439358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:29.439304    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:29.579198 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:29.579159    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:30.272388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:30.272348    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:30.321195 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:30.321149    5057 patch_prober.go:29] interesting pod/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body=
Nov 01 22:39:30.321399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:30.321233    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused"
Nov 01 22:39:30.999756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:30.999464    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:31.324366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:31.324290    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:39:31.325474 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:31.325452    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:39:31.799589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:31.799552    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:32.673076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:32.673039    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:33.456744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:33.456709    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:34.090623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:34.090581    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:34.806411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:34.806373    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:35.483239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:35.483204    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:36.109348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:36.109314    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:36.293351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:36.293314    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:36.986373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:36.985651    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:37.287750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:37.287661    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:37.437514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:37.429487    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:37.828481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:37.828442    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:38.506799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:38.506759    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:39.205621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:39.205579    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:39.883406 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:39.883363    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:40.021585 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:39:40.021540    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:39:40.572430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:40.572389    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:41.410237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:41.410195    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:42.030234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:42.030200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:42.240363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:42.240321    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:42.487069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:42.487032    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:42.616123 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:42.616085    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:42.972985 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:42.972941    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:43.533197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:43.533158    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:44.385964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:44.385927    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:44.806072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:44.806036    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:45.105980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:45.105883    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:45.876851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:45.876811    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:46.484350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:46.484267    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:46.675388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:46.675355    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:39:47.346956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:47.346879    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:48.304040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:48.304007    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:49.164320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:49.164284    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:50.202087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:50.202051    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:50.374036 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:50.374001    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:50.532298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:50.532198    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:50.787578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:50.774657    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:51.454240 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:51.454190    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:51.750429 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:51.736421    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:52.022588 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:52.022474    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:52.214945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:52.214901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:52.513712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:52.513658    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:53.081363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:53.081226    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:53.772666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:53.772630    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:54.649989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:54.649943    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:55.408301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:55.408248    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:56.396003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:56.395967    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:56.833981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:56.833944    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:56.980071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:56.980023    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:57.297450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:57.297413    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:58.034037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:58.033995    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:58.192517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:58.191284    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:58.591120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:58.591063    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:39:59.294381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:39:59.294337    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:00.068735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:00.068680    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:00.752117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:00.752083    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:01.445516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:01.441827    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:02.085901 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:02.085868    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:02.807856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:02.807821    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:02.955301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:02.955209    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:03.109635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:03.109550    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:03.283117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:03.283047    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:03.921554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:03.921523    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:04.574366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:04.574326    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:05.339741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:05.339696    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:06.281901 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:06.281862    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:07.150052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:07.150006    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:07.302534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:07.302476    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:07.584347 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:07.584311    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:08.371799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:08.371750    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:08.774382 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:08.774350    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:08.934003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:08.933952    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:09.496546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:09.496204    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:10.017558 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:40:10.017513    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:40:10.205893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:10.205852    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:10.986728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:10.986691    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:11.740482 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:11.740427    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:12.364472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:12.364431    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:12.995297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:12.995264    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:13.115699 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:13.115667    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:13.253656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:13.253569    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:13.676463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:13.676378    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:14.347158 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:14.345312    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:14.965020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:14.964976    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:15.674916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:15.674881    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:16.473247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:16.473212    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:17.115016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.114983    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"]
Nov 01 22:40:17.120806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.120777    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.132969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.132937    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"]
Nov 01 22:40:17.133576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.133549    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w6-c9a7ce-serving-cert\""
Nov 01 22:40:17.222793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.222735    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:17.315343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.315306    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.315582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.315420    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.315582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.315510    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6slhb\" (UniqueName: \"kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.315823 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.315797    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.368021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.367931    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:17.417290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.417250    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6slhb\" (UniqueName: \"kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.417483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.417379    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.417630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.417613    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.417695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.417652    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.417810 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:40:17.417793    5057 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-serving-cert: secret "maas-api-e2e-shared-a-w6-c9a7ce-serving-cert" not found
Nov 01 22:40:17.417882 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:40:17.417862    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls podName:c8b7ba43-4774-4977-a0db-1473f20b138b nodeName:}" failed. No retries permitted until 2026-11-01 22:40:17.917839847 +0000 UTC m=+1074.659535459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls") pod "maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" (UID: "c8b7ba43-4774-4977-a0db-1473f20b138b") : secret "maas-api-e2e-shared-a-w6-c9a7ce-serving-cert" not found
Nov 01 22:40:17.418206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.418179    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.420410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.420378    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.426704 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.426678    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6slhb\" (UniqueName: \"kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.641295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.641177    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:17.940690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.940058    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:17.962160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:17.961179    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:18.051318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.051277    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:18.177384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.177272    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:18.343261 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.343230    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"]
Nov 01 22:40:18.343869 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:40:18.343846    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8b7ba43_4774_4977_a0db_1473f20b138b.slice/crio-620a1245619549020004386368ec89cc2fc0c9113e1fadce3604280d0d4327cb WatchSource:0}: Error finding container 620a1245619549020004386368ec89cc2fc0c9113e1fadce3604280d0d4327cb: Status 404 returned error can't find the container with id 620a1245619549020004386368ec89cc2fc0c9113e1fadce3604280d0d4327cb
Nov 01 22:40:18.848010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.847969    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" event={"ID":"c8b7ba43-4774-4977-a0db-1473f20b138b","Type":"ContainerStarted","Data":"dda2c9e9d273d83f7eb31da7761ebd7adcb5a8d74f48041b0a6dcb97f2b41250"}
Nov 01 22:40:18.848479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.848030    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" event={"ID":"c8b7ba43-4774-4977-a0db-1473f20b138b","Type":"ContainerStarted","Data":"620a1245619549020004386368ec89cc2fc0c9113e1fadce3604280d0d4327cb"}
Nov 01 22:40:18.848479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.848065    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:18.859669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.859636    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:18.870729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:18.870682    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" podStartSLOduration=1.460783322 podStartE2EDuration="1.87066855s" totalImagesPullingTime="409.885228ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:40:18.866789518 +0000 UTC m=+1075.608485137" watchObservedRunningTime="2026-11-01 22:40:18.87066855 +0000 UTC m=+1075.612364200"
Nov 01 22:40:19.975630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:19.975573    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:20.772532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:20.770859    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:21.426180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:21.417637    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:22.032343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:22.032300    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:22.264047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:22.263606    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:22.471439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:22.471406    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:22.934570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:22.934196    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:23.104516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:23.099146    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:24.135435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:24.134470    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:24.873276 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:24.873241    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:40:24.888392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:24.888354    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:25.659518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:25.656922    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:26.585147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:26.585112    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:27.077947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:27.077910    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:27.232568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:27.232529    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:27.681458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:27.680174    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:28.485423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:28.485386    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:28.792565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:28.783462    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:28.939419 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:28.939378    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:29.115947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:29.115853    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:29.556306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:29.556108    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:29.999183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:29.999155    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:30.233420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:30.233371    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:31.113174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:31.113132    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:31.958515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:31.953540    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:33.051047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:33.050933    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:33.345122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:33.345015    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:33.486687 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:33.486650    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:33.980424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:33.969638    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:34.701240 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:34.701206    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:34.997567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:34.935829    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:35.394681 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:35.394593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:36.112869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:36.112831    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:36.886743 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:36.886705    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:37.396115 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:37.396060    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:37.562057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:37.562024    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:37.859179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:37.859139    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:38.188592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.188463    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"]
Nov 01 22:40:38.194755 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.194730    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.199867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.199845    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w6-3fa400-openshift-default-dockercfg-tfwlm\""
Nov 01 22:40:38.205384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.205359    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"]
Nov 01 22:40:38.281990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.281843    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.291299    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.291438    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.291676    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.291763    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.291845    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkfxv\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.291992    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.292090    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.292180    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.292612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.292219    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.403598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.403553    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.404259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.404236    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.404410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.404394    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkfxv\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.407703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.407678    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.408349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.408330    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.408521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.408481    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.409069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.409049    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.409329 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.409306    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.512811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.512778    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.513351    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.404560    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.404297    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.448211    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.451079    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.461208    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkfxv\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.513632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.409004    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.514113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.415272    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.545537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.544342    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token\") pod \"e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") " pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.710738 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.707630    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:38.842271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.842186    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:38.856948 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:38.856913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:39.237227 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:40:39.237175    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda18890f8_c3a4_4b2b_a261_983211e39f49.slice/crio-741da8eb4d2a30b713432df735987a7ddec84c87cb639f98aa3d320d2ad68d06 WatchSource:0}: Error finding container 741da8eb4d2a30b713432df735987a7ddec84c87cb639f98aa3d320d2ad68d06: Status 404 returned error can't find the container with id 741da8eb4d2a30b713432df735987a7ddec84c87cb639f98aa3d320d2ad68d06
Nov 01 22:40:39.238609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:39.238580    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"]
Nov 01 22:40:39.241160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:39.241116    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:40:39.241263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:39.241209    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:40:39.241263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:39.241252    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:40:40.033515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.031127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:40.081743 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:40:40.081701    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:40:40.202884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.202845    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" event={"ID":"a18890f8-c3a4-4b2b-a261-983211e39f49","Type":"ContainerStarted","Data":"4b579cb8778e742901c7037755e24861587b702e993695d5f3ba712ffd612695"}
Nov 01 22:40:40.203094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.202948    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" event={"ID":"a18890f8-c3a4-4b2b-a261-983211e39f49","Type":"ContainerStarted","Data":"741da8eb4d2a30b713432df735987a7ddec84c87cb639f98aa3d320d2ad68d06"}
Nov 01 22:40:40.234347 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.234273    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podStartSLOduration=2.234254814 podStartE2EDuration="2.234254814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:40:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:40:40.229653755 +0000 UTC m=+1096.971349371" watchObservedRunningTime="2026-11-01 22:40:40.234254814 +0000 UTC m=+1096.975950433"
Nov 01 22:40:40.844504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.842542    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:40.865557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.864660    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:40.873764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:40.873733    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:41.211263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:41.211200    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:41.212337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:41.212321    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:40:41.943479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:41.943116    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:42.145310 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:42.145269    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:42.343911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:42.343873    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:42.563819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:42.563785    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:42.723584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:42.723549    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:43.347610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:43.347576    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:43.895629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:43.895593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:44.115729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:44.115691    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:44.667761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:44.667717    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:45.597312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:45.597273    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:46.403166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:46.403085    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:47.136560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:47.136523    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:47.788246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:47.788214    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:48.255781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.255732    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:48.380186 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.380140    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:48.743462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.738478    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:48.850100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.850053    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"]
Nov 01 22:40:48.857514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.857061    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:48.875754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.875720    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"]
Nov 01 22:40:48.875998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.875847    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:48.882766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.882741    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a1d4be-serving-cert\""
Nov 01 22:40:48.943693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.943655    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:48.943910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.943753    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:48.943975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.943952    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:48.944054 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:48.944036    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4f6t\" (UniqueName: \"kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.046126 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.046040    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.046126 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.046117    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.046358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.046200    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.046358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.046248    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4f6t\" (UniqueName: \"kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.046818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.046788    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.048744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.048715    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.048849 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.048749    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.055251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.055226    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4f6t\" (UniqueName: \"kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t\") pod \"maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.124316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.124281    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:49.197538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.197502    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:49.607826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.607732    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"]
Nov 01 22:40:49.688925 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:40:49.687331    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d559e1_a742_4dde_b035_ba712f750510.slice/crio-7a7e93ca0001bc79f71f466240e6c708566f1426465662ee2442466e296f91d9 WatchSource:0}: Error finding container 7a7e93ca0001bc79f71f466240e6c708566f1426465662ee2442466e296f91d9: Status 404 returned error can't find the container with id 7a7e93ca0001bc79f71f466240e6c708566f1426465662ee2442466e296f91d9
Nov 01 22:40:49.969632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:49.969599    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:50.361259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:50.361171    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw" event={"ID":"b1d559e1-a742-4dde-b035-ba712f750510","Type":"ContainerStarted","Data":"5ee26d67034b03523c455c2d60ad35bfb1f47465071d4e77ce01c7daecfe1095"}
Nov 01 22:40:50.361259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:50.361237    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw" event={"ID":"b1d559e1-a742-4dde-b035-ba712f750510","Type":"ContainerStarted","Data":"7a7e93ca0001bc79f71f466240e6c708566f1426465662ee2442466e296f91d9"}
Nov 01 22:40:50.361482 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:50.361417    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:50.378231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:50.378169    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw" podStartSLOduration=1.985888923 podStartE2EDuration="2.378148505s" totalImagesPullingTime="392.259582ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:40:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:40:50.376702091 +0000 UTC m=+1107.118397709" watchObservedRunningTime="2026-11-01 22:40:50.378148505 +0000 UTC m=+1107.119844122"
Nov 01 22:40:50.851990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:50.851954    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:51.490773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:51.490735    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:52.109749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:52.109706    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:52.797038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:52.796994    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:53.507569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:53.506955    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:54.278866 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:54.278828    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:55.025792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:55.025743    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:55.760664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:55.760621    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:56.376090 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.376063    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:40:56.456538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.455801    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"]
Nov 01 22:40:56.475770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.475733    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"]
Nov 01 22:40:56.475967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.475862    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.486575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.483757    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w6-3fa400-serving-cert\""
Nov 01 22:40:56.504172 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.504116    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:56.627624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.627443    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:56.645748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.645712    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.645955 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.645893    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.647571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.646251    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd4d4\" (UniqueName: \"kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.647571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.646368    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.747367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.747330    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.747616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.747558    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.747818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.747798    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.747921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.747901    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd4d4\" (UniqueName: \"kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.748331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.748304    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.750857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.750834    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.751270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.751255    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.754569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.754543    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:56.757122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.757100    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd4d4\" (UniqueName: \"kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4\") pod \"maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:56.799388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:56.799344    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:57.316069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:57.316035    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"]
Nov 01 22:40:57.317331 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:40:57.317307    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad6eb752_b2b7_4d80_8ced_a98d09c0c1bd.slice/crio-1165b27bbf55023f2f55bd348984e48ead527f0b413159396d2a6ed2676e5e8f WatchSource:0}: Error finding container 1165b27bbf55023f2f55bd348984e48ead527f0b413159396d2a6ed2676e5e8f: Status 404 returned error can't find the container with id 1165b27bbf55023f2f55bd348984e48ead527f0b413159396d2a6ed2676e5e8f
Nov 01 22:40:57.447578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:57.447540    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb" event={"ID":"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd","Type":"ContainerStarted","Data":"1165b27bbf55023f2f55bd348984e48ead527f0b413159396d2a6ed2676e5e8f"}
Nov 01 22:40:57.489753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:57.489716    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:58.516558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:58.511403    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb" event={"ID":"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd","Type":"ContainerStarted","Data":"127dd2b493855314e7968468d0f9534a9894f06702c3581f036c0c79b1b4bd19"}
Nov 01 22:40:58.522281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:58.520572    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:40:58.558302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:58.558226    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb" podStartSLOduration=2.087387781 podStartE2EDuration="2.558203748s" totalImagesPullingTime="470.815967ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:40:58.549920146 +0000 UTC m=+1115.291615766" watchObservedRunningTime="2026-11-01 22:40:58.558203748 +0000 UTC m=+1115.299899367"
Nov 01 22:40:58.781603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:58.781467    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:59.738932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:59.738889    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:40:59.889714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:40:59.889681    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:00.513940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:00.513897    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:00.692812 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:00.692645    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:01.439125 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:01.439081    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:02.181113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:02.181081    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:02.572566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:02.572528    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:02.991616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:02.991576    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:03.609516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:03.605039    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:04.220696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:04.220659    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:04.453518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:04.432127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:04.590850 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:04.590742    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:41:05.325518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:05.312946    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:05.508589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:05.508551    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:41:06.307787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:06.307738    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:07.136706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:07.136659    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:08.403278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:08.403227    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:08.563558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:08.563511    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:08.760200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:08.760154    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:09.076558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:09.076437    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:09.400936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:09.399750    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:10.138567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:10.129331    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:10.419637 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:41:10.419486    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:41:10.475966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:10.475932    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:10.616053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:10.616015    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:10.859596 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:10.859565    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:12.163836 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:12.163791    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:12.986691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:12.986656    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:13.184171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:13.184136    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:13.671986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:13.671945    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:14.728878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:14.728844    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:15.050756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:15.050675    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:15.258098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:15.258060    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:16.260593 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:16.260555    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:16.470391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:16.470324    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:17.356183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:17.356126    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:17.502987 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:17.502924    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:17.654646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:17.653368    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:18.009563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:17.992454    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:18.116615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:18.116527    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:18.243523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:18.243478    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:18.366384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:18.366288    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:19.055455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:19.055414    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:19.487383 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:19.487266    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:19.797226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:19.797140    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:20.129334 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:20.129245    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:20.278931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:20.278800    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:20.782462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:20.782411    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:21.685923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:21.685884    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:22.007900 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:22.007865    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:22.474223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:22.474184    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:23.217996 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:23.217949    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:24.096504 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:24.096455    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:24.306933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:24.306894    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:24.464101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:24.464057    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:25.304581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:25.304543    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:25.465908 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:25.465879    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:25.609135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:25.608932    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:26.135930 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:26.135894    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:26.336540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:26.332918    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:26.747514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:26.738816    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:26.918139 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:26.918084    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:27.070060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.069962    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:27.205967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.205901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:27.597724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.597681    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"]
Nov 01 22:41:27.604673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.604647    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.609807 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.609778    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"]
Nov 01 22:41:27.626920 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.626890    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-ee8d1a-serving-cert\""
Nov 01 22:41:27.723733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.716538    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.723733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.717167    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.723733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.717555    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.723733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.717784    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q8n5\" (UniqueName: \"kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.743692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.742801    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:27.826059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.824092    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.826059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.824152    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q8n5\" (UniqueName: \"kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.826059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.824360    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.826059 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.824461    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.832188 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:41:27.832142    5057 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-serving-cert: secret "maas-api-e2e-api-a-ee8d1a-serving-cert" not found
Nov 01 22:41:27.832375 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:41:27.832235    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls podName:ca20de96-6023-4e74-b8f1-d65d2123e913 nodeName:}" failed. No retries permitted until 2026-11-01 22:41:28.332201269 +0000 UTC m=+1145.073896866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls") pod "maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" (UID: "ca20de96-6023-4e74-b8f1-d65d2123e913") : secret "maas-api-e2e-api-a-ee8d1a-serving-cert" not found
Nov 01 22:41:27.833213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.833177    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.841169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.841134    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.853480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.853391    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q8n5\" (UniqueName: \"kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:27.886889 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:27.886856    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:28.038477 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.038443    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:28.339527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.338468    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:28.348404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.347445    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\") pod \"maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:28.413542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.412067    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:28.643917 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.643826    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:28.723320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.723000    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:28.964916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.964883    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"]
Nov 01 22:41:28.965324 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:41:28.965177    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca20de96_6023_4e74_b8f1_d65d2123e913.slice/crio-9ce840c2f0c5ef5e4841298b568824d3624adfc1c2b8547a88161d47ab962c63 WatchSource:0}: Error finding container 9ce840c2f0c5ef5e4841298b568824d3624adfc1c2b8547a88161d47ab962c63: Status 404 returned error can't find the container with id 9ce840c2f0c5ef5e4841298b568824d3624adfc1c2b8547a88161d47ab962c63
Nov 01 22:41:28.967143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:28.967120    5057 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Nov 01 22:41:29.239063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.238982    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" event={"ID":"ca20de96-6023-4e74-b8f1-d65d2123e913","Type":"ContainerStarted","Data":"9ce840c2f0c5ef5e4841298b568824d3624adfc1c2b8547a88161d47ab962c63"}
Nov 01 22:41:29.508891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.508841    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:29.710055 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.710019    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"]
Nov 01 22:41:29.714835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.714813    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.726479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.725347    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a1d4be-openshift-default-dockercfg-2snjb\""
Nov 01 22:41:29.734769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.734705    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"]
Nov 01 22:41:29.781038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.780950    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"]
Nov 01 22:41:29.785949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.785909    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.786279 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786231    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.786464 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786448    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786624    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786669    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786696    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99zs\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786721    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786815    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.786843    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.787384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.787154    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.792215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.792057    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"]
Nov 01 22:41:29.888241 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888210    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.888399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888284    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.888399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888356    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888471 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888434    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888477    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888587    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888621    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888654    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k99zs\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888687    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.888746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888718    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888776    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.888966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.888927    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.889062 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889034    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.889131 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889118    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.889190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889148    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.889240 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889216    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.889277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889248    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjnbh\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.889316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889287    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.889347 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889324    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.889384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889365    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.889436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889410    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.889654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.889628    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.890081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.890053    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.891866 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.891845    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.894375 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.894352    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.901649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.901618    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.902163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.902139    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99zs\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:29.991046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991009    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991193    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991336    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991366    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991419    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjnbh\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991460    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991507 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991462    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991564    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991570    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991635    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991715    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.991830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.991738    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.992061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.992042    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.992750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.992724    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.994202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.994180    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.995128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.995108    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.999785 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.999759    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:29.999963 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:29.999944    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjnbh\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:30.050685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.050595    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:30.103513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.101908    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:30.289190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.289149    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" event={"ID":"ca20de96-6023-4e74-b8f1-d65d2123e913","Type":"ContainerStarted","Data":"3287ec2727966e43f5361bd0a2b71cdd9b63af963717ec3ff38d959095c66478"}
Nov 01 22:41:30.290405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.290383    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:30.320845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.320730    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" podStartSLOduration=2.79383517 podStartE2EDuration="3.320713768s" totalImagesPullingTime="526.878598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:41:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:41:30.317902361 +0000 UTC m=+1147.059597979" watchObservedRunningTime="2026-11-01 22:41:30.320713768 +0000 UTC m=+1147.062409385"
Nov 01 22:41:30.398796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.398769    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"]
Nov 01 22:41:30.399269 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:41:30.399246    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2ab5df1_4570_4aed_a39f_c3faee5a8df9.slice/crio-2fcb7f450b65a04125cc50ae2328e3ebda72eb7237e6c1116ae92122b61689a8 WatchSource:0}: Error finding container 2fcb7f450b65a04125cc50ae2328e3ebda72eb7237e6c1116ae92122b61689a8: Status 404 returned error can't find the container with id 2fcb7f450b65a04125cc50ae2328e3ebda72eb7237e6c1116ae92122b61689a8
Nov 01 22:41:30.402254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.402213    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:41:30.402340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.402297    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:41:30.402340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.402327    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:41:30.470337 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:41:30.470304    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7292edd4_bdf6_4f1e_be5d_f41670c8a84f.slice/crio-5509692c0f7a838fb96a2c2a29b6d023bfc5d009dc86fa289dc57d78b04d298a WatchSource:0}: Error finding container 5509692c0f7a838fb96a2c2a29b6d023bfc5d009dc86fa289dc57d78b04d298a: Status 404 returned error can't find the container with id 5509692c0f7a838fb96a2c2a29b6d023bfc5d009dc86fa289dc57d78b04d298a
Nov 01 22:41:30.471288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.471259    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"]
Nov 01 22:41:30.473988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.473946    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:41:30.474076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.474017    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:41:30.474076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.474060    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:41:30.532401 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:30.528073    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:31.299947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.299909    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" event={"ID":"b2ab5df1-4570-4aed-a39f-c3faee5a8df9","Type":"ContainerStarted","Data":"42e84539a8fb89d414a3c79ab9689fc9780667692458c60796b94f24e58c5dcc"}
Nov 01 22:41:31.300577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.299974    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" event={"ID":"b2ab5df1-4570-4aed-a39f-c3faee5a8df9","Type":"ContainerStarted","Data":"2fcb7f450b65a04125cc50ae2328e3ebda72eb7237e6c1116ae92122b61689a8"}
Nov 01 22:41:31.302027 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.302000    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" event={"ID":"7292edd4-bdf6-4f1e-be5d-f41670c8a84f","Type":"ContainerStarted","Data":"5d5edab34004c55f36f0ffc2dd1adcb22859bd6e2feb80147b052587e0b839ba"}
Nov 01 22:41:31.302128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.302049    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" event={"ID":"7292edd4-bdf6-4f1e-be5d-f41670c8a84f","Type":"ContainerStarted","Data":"5509692c0f7a838fb96a2c2a29b6d023bfc5d009dc86fa289dc57d78b04d298a"}
Nov 01 22:41:31.347220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.347097    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podStartSLOduration=2.347077579 podStartE2EDuration="2.347077579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:41:31.33471326 +0000 UTC m=+1148.076408881" watchObservedRunningTime="2026-11-01 22:41:31.347077579 +0000 UTC m=+1148.088773199"
Nov 01 22:41:31.366873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.366819    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podStartSLOduration=2.366804274 podStartE2EDuration="2.366804274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:41:31.366134598 +0000 UTC m=+1148.107830216" watchObservedRunningTime="2026-11-01 22:41:31.366804274 +0000 UTC m=+1148.108499891"
Nov 01 22:41:31.777951 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:31.776055    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:32.060579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.054897    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:32.060579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.055111    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 01 22:41:32.060579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.055156    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 01 22:41:32.102828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.102796    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:32.104261 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.104009    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 01 22:41:32.104261 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.104057    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 01 22:41:32.642754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:32.642686    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:33.098658 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:33.098622    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 01 22:41:33.099010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:33.098980    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 01 22:41:33.109381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:33.102708    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 01 22:41:33.109381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:33.102756    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 01 22:41:33.509923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:33.509886    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:34.059966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:34.058571    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 01 22:41:34.059966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:34.058627    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 01 22:41:34.105097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:34.105049    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 01 22:41:34.105445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:34.105415    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 01 22:41:34.406110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:34.405985    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:34.560640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:34.552453    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:35.061515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:35.060927    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 01 22:41:35.061515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:35.060999    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 01 22:41:35.102778 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:35.102747    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 01 22:41:35.102964 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:35.102802    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 01 22:41:35.290030 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:35.289910    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:36.054516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.054103    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 01 22:41:36.054516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.054166    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 01 22:41:36.090516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.089865    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"]
Nov 01 22:41:36.107187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.105849    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 01 22:41:36.107187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.105932    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 01 22:41:36.115656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.111953    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"]
Nov 01 22:41:36.115656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.112068    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.171518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.137432    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b4f6bb-serving-cert\""
Nov 01 22:41:36.196993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.140634    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:36.207387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.207360    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.207563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.207407    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.207563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.207463    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.207680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.207662    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjjk5\" (UniqueName: \"kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.327095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.326924    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.327391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.327363    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.327793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.327659    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.328257 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.328234    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjjk5\" (UniqueName: \"kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.360959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.360922    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.361727 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.361695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:36.364002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.363977    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.590369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.590256    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjjk5\" (UniqueName: \"kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.608174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.608140    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs\") pod \"maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:36.799144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:36.798830    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:37.054515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.053789    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body=
Nov 01 22:41:37.054515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.053848    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused"
Nov 01 22:41:37.124522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.122913    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused" start-of-body=
Nov 01 22:41:37.124522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.123012    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.99:15021/healthz/ready\": dial tcp 10.132.0.99:15021: connect: connection refused"
Nov 01 22:41:37.387568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.386230    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:41:37.409122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.408954    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:37.583663 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:41:37.583618    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8aa617_b822_4f46_8bb2_6bdb371b9982.slice/crio-ac50fe6494a7f3030b937a921627e7d9401c433dc1fe6798810baf89bbb7412e WatchSource:0}: Error finding container ac50fe6494a7f3030b937a921627e7d9401c433dc1fe6798810baf89bbb7412e: Status 404 returned error can't find the container with id ac50fe6494a7f3030b937a921627e7d9401c433dc1fe6798810baf89bbb7412e
Nov 01 22:41:37.599102 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:37.596940    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"]
Nov 01 22:41:38.060249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.060222    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:38.061138 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.061086    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:38.062461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.062420    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:41:38.112058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.111199    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:38.112058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.111553    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:38.112563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.112542    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:41:38.182528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.174200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:38.331232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.330376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:38.557610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.557573    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw" event={"ID":"cd8aa617-b822-4f46-8bb2-6bdb371b9982","Type":"ContainerStarted","Data":"c713593ad372124973b406f86e4c8cba1664fa5287317a2db7f5146bfde830d5"}
Nov 01 22:41:38.557821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.557625    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw" event={"ID":"cd8aa617-b822-4f46-8bb2-6bdb371b9982","Type":"ContainerStarted","Data":"ac50fe6494a7f3030b937a921627e7d9401c433dc1fe6798810baf89bbb7412e"}
Nov 01 22:41:38.558656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.558632    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:38.584295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:38.584197    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw" podStartSLOduration=2.132014309 podStartE2EDuration="2.584168678s" totalImagesPullingTime="452.154369ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:41:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:41:38.577863889 +0000 UTC m=+1155.319559507" watchObservedRunningTime="2026-11-01 22:41:38.584168678 +0000 UTC m=+1155.325864299"
Nov 01 22:41:39.297692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:39.297654    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:40.016321 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:41:40.016287    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:41:40.077459 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:40.076261    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:40.841226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:40.841181    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:41.565134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:41.565084    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:42.265013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:42.264969    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:42.968408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:42.968273    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:43.080096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:43.080059    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:43.375988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:43.375858    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:43.893402 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:43.893366    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:44.583840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:44.583802    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:45.378752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:45.378718    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:45.579178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:45.579149    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:41:46.250316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:46.250279    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:46.814917 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:46.814881    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:47.797324 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:47.797287    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:48.797308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:48.797269    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:49.122946 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.122868    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-kbdg8-jm4bc"]
Nov 01 22:41:49.128630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.128607    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.134295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.134273    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-dc544\""
Nov 01 22:41:49.138183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.138163    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-kbdg8-jm4bc"]
Nov 01 22:41:49.287517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.281832    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nq95\" (UniqueName: \"kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.287517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.282095    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.287517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.282176    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.287517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.282240    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.425575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.387707    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.461861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.461818    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.462269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.462248    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.462685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.462649    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nq95\" (UniqueName: \"kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.466395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.466373    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.474461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.474437    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.490845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.490811    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.499004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.492593    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nq95\" (UniqueName: \"kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95\") pod \"periodic-gathering-kbdg8-jm4bc\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") " pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.512662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.512634    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:41:49.524483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.524453    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:41:49.843801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:49.843775    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-kbdg8-jm4bc"]
Nov 01 22:41:49.850147 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:41:49.850112    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5214431e_5fc7_4ec2_af7c_e32a789c7d0f.slice/crio-617f6341e36e1054c99ff593d8f157e28a7ec0dcbe23a2092328401c4f2ccb45 WatchSource:0}: Error finding container 617f6341e36e1054c99ff593d8f157e28a7ec0dcbe23a2092328401c4f2ccb45: Status 404 returned error can't find the container with id 617f6341e36e1054c99ff593d8f157e28a7ec0dcbe23a2092328401c4f2ccb45
Nov 01 22:41:50.703045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:50.702996    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc" event={"ID":"5214431e-5fc7-4ec2-af7c-e32a789c7d0f","Type":"ContainerStarted","Data":"5ff95fa5ed6fbb4c45cd6b93d3f0e5cc74a73e2e720f8387b4465ec58ddf6737"}
Nov 01 22:41:50.703571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:50.703062    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc" event={"ID":"5214431e-5fc7-4ec2-af7c-e32a789c7d0f","Type":"ContainerStarted","Data":"617f6341e36e1054c99ff593d8f157e28a7ec0dcbe23a2092328401c4f2ccb45"}
Nov 01 22:41:50.730187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:50.730122    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc" podStartSLOduration=1.7301044220000001 podStartE2EDuration="1.730104422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:41:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:41:50.726645326 +0000 UTC m=+1167.468340946" watchObservedRunningTime="2026-11-01 22:41:50.730104422 +0000 UTC m=+1167.471800042"
Nov 01 22:41:51.089560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:51.088959    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:52.218514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:52.209459    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:53.267621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.267574    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded"
Nov 01 22:41:53.371636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.371580    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": context deadline exceeded"
Nov 01 22:41:53.656570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.655574    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": context deadline exceeded"
Nov 01 22:41:53.656752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.656592    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:53.680516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.679167    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:53.788518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.787766    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:41:53.883081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:53.876879    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:54.341711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.341662    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:54.342871 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.342844    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:41:54.538025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.537976    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:54.654568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.643318    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:41:54.654568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.650733    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:41:54.656447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.656413    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:54.995963 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:54.788082    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:55.125519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.016039    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:41:55.125519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.119072    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:41:55.125519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.119139    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:41:55.162515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.099725    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:55.374709 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.373847    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:55.719680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.681623    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:55.719680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.682303    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded"
Nov 01 22:41:55.719680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.681557    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:55.776583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.771579    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:55.804937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:55.804901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:56.101570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.010070    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:41:56.101570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.010131    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:41:56.101570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.072982    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:41:56.187559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.187521    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:41:56.188467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.188300    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:41:56.204510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.203745    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:41:56.505780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:56.505740    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:57.496854 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:57.496811    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:58.810533 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:58.810392    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:41:59.951482 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:41:59.951442    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:00.809358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:00.809276    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:01.691763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:01.691711    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:02.379293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:02.379251    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:03.156335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.156252    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:03.295748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.293341    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:03.342174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.342138    5057 generic.go:356] "Generic (PLEG): container finished" podID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerID="4b579cb8778e742901c7037755e24861587b702e993695d5f3ba712ffd612695" exitCode=0
Nov 01 22:42:03.342381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.342362    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" event={"ID":"a18890f8-c3a4-4b2b-a261-983211e39f49","Type":"ContainerDied","Data":"4b579cb8778e742901c7037755e24861587b702e993695d5f3ba712ffd612695"}
Nov 01 22:42:03.342943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.342916    5057 scope.go:122] "RemoveContainer" containerID="4b579cb8778e742901c7037755e24861587b702e993695d5f3ba712ffd612695"
Nov 01 22:42:03.345562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.345461    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:03.345688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.345563    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:03.345688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:03.345613    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:04.097180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:04.097064    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:04.380741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:04.380658    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" event={"ID":"a18890f8-c3a4-4b2b-a261-983211e39f49","Type":"ContainerStarted","Data":"488fed8c6cfa120951584bdb26f9608f3e51e9851c7ec525e389ee46f3a792b1"}
Nov 01 22:42:04.853612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:04.845071    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:42:04.853612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:04.845204    5057 patch_prober.go:29] interesting pod/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 01 22:42:04.853612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:04.845281    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 01 22:42:04.973515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:04.971333    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:05.148587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:05.148484    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:05.858547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:05.855637    5057 patch_prober.go:29] interesting pod/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 01 22:42:05.858547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:05.855704    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 01 22:42:06.842878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:06.842843    5057 patch_prober.go:29] interesting pod/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 01 22:42:06.843209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:06.843180    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 01 22:42:06.882218 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:06.882180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:07.845904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:07.845867    5057 patch_prober.go:29] interesting pod/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 01 22:42:07.846179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:07.846149    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 01 22:42:08.257520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:08.257164    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:08.842891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:08.842857    5057 patch_prober.go:29] interesting pod/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 01 22:42:08.843214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:08.843184    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 01 22:42:09.316265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:09.315296    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:09.843230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:09.843194    5057 patch_prober.go:29] interesting pod/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Nov 01 22:42:09.843405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:09.843262    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Nov 01 22:42:09.942356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:09.942320    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:10.021860 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:42:10.021822    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:42:10.109363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:10.109275    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:10.461380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:10.461336    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:10.848391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:10.848299    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:42:10.848748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:10.848727    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:42:10.849745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:10.849727    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:42:11.676299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:11.676260    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:12.059666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:12.051165    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:12.721568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:12.715276    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:13.197212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:13.197131    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:13.670399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:13.670364    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:14.410266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:14.410180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:15.329227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:15.329193    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"]
Nov 01 22:42:15.329470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:15.329442    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy" containerID="cri-o://32102371113d6862e7e257b4a86c2c1f90c6e744413df740dab2114bb682c297" gracePeriod=30
Nov 01 22:42:15.335470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:15.335443    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"]
Nov 01 22:42:15.335708 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:15.335686    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy" containerID="cri-o://9655b8594b2df34471894c360e505731b235f9d5642e79794211158f6ece300e" gracePeriod=30
Nov 01 22:42:15.542220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:15.542179    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:15.762698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:15.762661    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:16.239932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:16.239894    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:16.771885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:16.771850    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:17.597604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:17.591601    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:18.382522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:18.382456    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:19.371517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:19.349555    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:20.204084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.204045    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:20.798480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.796322    5057 generic.go:356] "Generic (PLEG): container finished" podID="3324062a-c889-4668-8c46-29ab982b259c" containerID="32102371113d6862e7e257b4a86c2c1f90c6e744413df740dab2114bb682c297" exitCode=0
Nov 01 22:42:20.798480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.796488    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" event={"ID":"3324062a-c889-4668-8c46-29ab982b259c","Type":"ContainerDied","Data":"32102371113d6862e7e257b4a86c2c1f90c6e744413df740dab2114bb682c297"}
Nov 01 22:42:20.799300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.799272    5057 generic.go:356] "Generic (PLEG): container finished" podID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerID="9655b8594b2df34471894c360e505731b235f9d5642e79794211158f6ece300e" exitCode=0
Nov 01 22:42:20.799453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.799352    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" event={"ID":"63b1d51d-190e-49ff-af02-f30851ac3d25","Type":"ContainerDied","Data":"9655b8594b2df34471894c360e505731b235f9d5642e79794211158f6ece300e"}
Nov 01 22:42:20.861995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.860563    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:42:20.959842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.959800    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.959958    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960075    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960118    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960190    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960245    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960282    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960446    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.960587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.960543    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket\") pod \"63b1d51d-190e-49ff-af02-f30851ac3d25\" (UID: \"63b1d51d-190e-49ff-af02-f30851ac3d25\") "
Nov 01 22:42:20.961486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.961458    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:20.964764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.964734    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:20.964939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.964920    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:20.965209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.965187    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:42:20.967192 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.967079    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:42:20.967192 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.967167    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:20.969622 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.969591    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:20.985032 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.984975    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "kube-api-access-6kksh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:42:20.986798 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.986336    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"]
Nov 01 22:42:20.987437 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.987410    5057 cpu_manager.go:414] "containerMap: removing container" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy"
Nov 01 22:42:20.987563 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.987439    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy"
Nov 01 22:42:20.987701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.987681    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" containerName="istio-proxy"
Nov 01 22:42:20.993524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.993487    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"]
Nov 01 22:42:20.993626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:20.993616    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.057868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.053736    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token" pod "63b1d51d-190e-49ff-af02-f30851ac3d25" (UID: "63b1d51d-190e-49ff-af02-f30851ac3d25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.061384    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.062646    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.062710    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.062747    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.062788    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.062817    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.062863    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063053    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kp4l\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063126    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063180    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063405    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063420    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063436    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063449    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63b1d51d-190e-49ff-af02-f30851ac3d25-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063462    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.077308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.063476    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kksh\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-kube-api-access-6kksh\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.084146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.080394    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63b1d51d-190e-49ff-af02-f30851ac3d25-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.084146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.080446    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.084146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.080462    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63b1d51d-190e-49ff-af02-f30851ac3d25-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.105559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.104027    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.182290    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.182428    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.182797    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kp4l\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.182877    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.182941    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.183154    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.183210    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.183246    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.183286    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.183317    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.183366    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.187447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.186460    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"]
Nov 01 22:42:21.188267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.187837    5057 cpu_manager.go:414] "containerMap: removing container" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy"
Nov 01 22:42:21.188267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.187856    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy"
Nov 01 22:42:21.188267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.188080    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="3324062a-c889-4668-8c46-29ab982b259c" containerName="istio-proxy"
Nov 01 22:42:21.188748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.188657    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.189210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.188923    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.192508 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.190761    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:21.192644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.191585    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.192737 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.192217    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.192819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.192333    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.192904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.192439    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.217519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.211665    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.237271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.237241    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kp4l\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.253268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.253203    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.253606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.253568    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:42:21.264146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.264070    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-b654v\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.268668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.268645    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"]
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295107    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295203    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295265    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295351    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295375    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295538    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295610    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data\") pod \"3324062a-c889-4668-8c46-29ab982b259c\" (UID: \"3324062a-c889-4668-8c46-29ab982b259c\") "
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.295853    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.296189    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.296419    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.296458    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.296475    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3324062a-c889-4668-8c46-29ab982b259c-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.297614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.296610    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.298356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.297655    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:21.298356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.297855    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:21.299239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.299193    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:42:21.299916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.299871    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "kube-api-access-p9td5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:42:21.301037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.301013    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo" pod "3324062a-c889-4668-8c46-29ab982b259c" (UID: "3324062a-c889-4668-8c46-29ab982b259c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.409560    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411091    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411137    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411178    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411219    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411291    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411329    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmkn6\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411372    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411510    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411633    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411840    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3324062a-c889-4668-8c46-29ab982b259c-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411860    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411875    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411888    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3324062a-c889-4668-8c46-29ab982b259c-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.412634 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.411904    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9td5\" (UniqueName: \"kubernetes.io/projected/3324062a-c889-4668-8c46-29ab982b259c-kube-api-access-p9td5\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:42:21.514217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514172    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514343    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514582    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514618    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514654    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514697    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514930    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.514999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.514989    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmkn6\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.515373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.515031    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.515568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.515543    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.516298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.515763    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.516298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.515945    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.516933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.516896    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.517160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.517134    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.522536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.517749    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.522536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.520119    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.528758 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.528708    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmkn6\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.529695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.529671    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token\") pod \"e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") " pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.542181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.541959    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:21.838196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.838158    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk" event={"ID":"3324062a-c889-4668-8c46-29ab982b259c","Type":"ContainerDied","Data":"5964d058104aca6e03c82d631c970167fc0e727f4d351363d39252b03022005c"}
Nov 01 22:42:21.838358 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.838225    5057 scope.go:122] "RemoveContainer" containerID="32102371113d6862e7e257b4a86c2c1f90c6e744413df740dab2114bb682c297"
Nov 01 22:42:21.838434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.838390    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"
Nov 01 22:42:21.844800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.843986    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"
Nov 01 22:42:21.844800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.843985    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq" event={"ID":"63b1d51d-190e-49ff-af02-f30851ac3d25","Type":"ContainerDied","Data":"7a2a5d3604a6a5472a0a765406aa6244fcefb114c1b61138cce7694104c14bfc"}
Nov 01 22:42:21.862306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.862268    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"]
Nov 01 22:42:21.865765 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.865740    5057 scope.go:122] "RemoveContainer" containerID="9655b8594b2df34471894c360e505731b235f9d5642e79794211158f6ece300e"
Nov 01 22:42:21.879853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.879822    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"]
Nov 01 22:42:21.885822 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.885764    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-cv9nk"]
Nov 01 22:42:21.890730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.889973    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:21.890730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.890067    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:21.890730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.890109    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:21.890730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.890361    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"]
Nov 01 22:42:21.896049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:21.895984    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-6xgtq"]
Nov 01 22:42:22.089098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.089053    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:22.089225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.089147    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:22.089225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.089191    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:42:22.096677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.096653    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"]
Nov 01 22:42:22.206837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.206033    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:22.876903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.876738    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" event={"ID":"e8586610-fbf5-4c12-a557-33747da52fc3","Type":"ContainerStarted","Data":"5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530"}
Nov 01 22:42:22.876903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.876805    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" event={"ID":"e8586610-fbf5-4c12-a557-33747da52fc3","Type":"ContainerStarted","Data":"d2773376a866bdd3615e17ae127cecc85b0787861b33b0f7faef9d37f59dd5bb"}
Nov 01 22:42:22.884034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.883431    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" event={"ID":"45732ef3-3377-4231-bb8b-54f77cb7cf5c","Type":"ContainerStarted","Data":"de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168"}
Nov 01 22:42:22.884034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.883516    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" event={"ID":"45732ef3-3377-4231-bb8b-54f77cb7cf5c","Type":"ContainerStarted","Data":"5c33f5a2e7dd3e0beaeca5a9d0d4555ee3958548acd384618e491ab44606a7d7"}
Nov 01 22:42:22.928821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.928762    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podStartSLOduration=8.928742606 podStartE2EDuration="8.928742606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:42:22.918956851 +0000 UTC m=+1199.660652471" watchObservedRunningTime="2026-11-01 22:42:22.928742606 +0000 UTC m=+1199.670438225"
Nov 01 22:42:22.944297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:22.944248    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podStartSLOduration=8.94423372 podStartE2EDuration="8.94423372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:42:22.942825096 +0000 UTC m=+1199.684520713" watchObservedRunningTime="2026-11-01 22:42:22.94423372 +0000 UTC m=+1199.685929337"
Nov 01 22:42:23.084091 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.084046    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:23.415814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.415778    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:23.415988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.415841    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:23.420839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.420808    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3324062a-c889-4668-8c46-29ab982b259c" path="/var/lib/kubelet/pods/3324062a-c889-4668-8c46-29ab982b259c/volumes"
Nov 01 22:42:23.421630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.421611    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63b1d51d-190e-49ff-af02-f30851ac3d25" path="/var/lib/kubelet/pods/63b1d51d-190e-49ff-af02-f30851ac3d25/volumes"
Nov 01 22:42:23.422101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.422083    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:23.555224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.554602    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:23.556414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.556113    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:23.556414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.556166    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:23.595371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.595345    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:42:23.600328 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.600303    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:42:23.775603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.775567    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:23.933547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:23.930831    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:24.048050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:24.046870    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:24.413605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:24.413522    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:24.413605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:24.413578    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:24.543356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:24.543323    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:24.543601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:24.543379    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:24.815151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:24.815114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:25.413330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:25.413298    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:25.413752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:25.413350    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:25.480479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:25.480433    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:25.543682 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:25.543643    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:25.543904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:25.543707    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:26.384618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:26.384582    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:26.413115 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:26.413084    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:26.413287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:26.413146    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:26.543339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:26.543307    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:26.543825 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:26.543362    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:27.061565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:27.061529    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:27.412757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:27.412678    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:27.412757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:27.412731    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:27.543574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:27.543535    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:27.544001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:27.543611    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:27.718089 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:27.718050    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:28.384268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.384234    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:28.413137 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.413102    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:28.413344 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.413171    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:28.510232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.510193    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:28.543348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.543303    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:28.543694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.543659    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:28.630202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:28.630164    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:29.199133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:29.199091    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:29.412554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:29.412519    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:29.412723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:29.412573    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:29.542630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:29.542594    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:29.542831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:29.542641    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:29.986199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:29.986165    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:30.413452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:30.413367    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:30.413452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:30.413429    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:30.543114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:30.543075    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:30.543297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:30.543150    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:30.697195 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:30.697112    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:31.413022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:31.412990    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:31.413420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:31.413060    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:31.501602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:31.501478    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:31.546919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:31.546883    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:31.547246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:31.547213    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:32.341205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:32.341161    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:32.413646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:32.413570    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:32.414145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:32.413672    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:32.542911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:32.542872    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:32.543150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:32.542943    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:33.188292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:33.188186    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:33.412742 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:33.412699    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:33.412936 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:33.412765    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:33.543250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:33.543215    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:33.543691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:33.543265    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:34.206132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:34.206069    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:34.413448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:34.413409    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:34.413662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:34.413465    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:34.542762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:34.542686    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:34.542762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:34.542739    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:35.036320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:35.036283    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:35.413330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:35.413250    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:35.413330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:35.413315    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:35.543450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:35.543410    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:35.543689 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:35.543471    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:36.072487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.072444    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:36.412680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.412606    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:36.412881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.412862    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:36.412937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.412907    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:36.543075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.543042    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:36.543325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.543303    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:36.543385 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.543355    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:36.790935 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:36.790893    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:37.235594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:37.235564    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:37.412530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:37.412481    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:37.412734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:37.412555    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:37.493520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:37.493393    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:37.543181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:37.543136    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:37.543367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:37.543224    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:38.344762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:38.344709    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:38.413076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:38.413042    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:38.413257 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:38.413099    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:38.543461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:38.543421    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:38.543670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:38.543488    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:38.933482 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:38.933437    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:39.413248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:39.413216    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:39.413672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:39.413276    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:39.542917 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:39.542878    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:39.543106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:39.542946    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:39.714751 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:39.713601    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:40.025462 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:42:40.025380    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:42:40.424571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:40.412744    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:40.424571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:40.412820    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:40.539161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:40.539110    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:40.547020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:40.546953    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:40.547723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:40.547267    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:41.413333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:41.413288    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:41.413545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:41.413357    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:41.440203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:41.440152    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:41.547638 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:41.547603    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:41.548142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:41.548109    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:42.260516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:42.260371    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:42.413076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:42.413033    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:42.413294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:42.413097    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:42.545572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:42.544876    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:42.545572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:42.544938    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:43.412583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:43.412458    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:43.412864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:43.412837    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:43.457514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:43.451479    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:43.545517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:43.544981    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:43.545517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:43.545039    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:44.398514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:44.395534    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:44.416488 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:44.413062    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:44.416488 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:44.413117    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:44.561517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:44.548099    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:44.561517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:44.548178    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:44.787141 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:44.787103    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:45.396020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:45.395989    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:45.424520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:45.419914    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body=
Nov 01 22:42:45.425475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:45.425418    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused"
Nov 01 22:42:45.550516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:45.549947    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body=
Nov 01 22:42:45.550516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:45.550020    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused"
Nov 01 22:42:46.385691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:46.385642    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:46.512079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:46.512049    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:46.520234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:46.516202    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:42:46.558816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:46.556154    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:46.558816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:46.558244    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:42:47.291230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:47.291198    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:48.042842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:48.042800    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:48.886160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:48.886117    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:49.670273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:49.670228    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:50.458890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:50.458843    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:51.167164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:51.167127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:51.931906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:51.931866    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:52.571853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:52.571819    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:53.259162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:53.259128    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:54.047092 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:54.047058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:54.957692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:54.957649    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:55.704361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:55.704328    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:56.393128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:56.393084    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:57.153221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:57.153185    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:57.274399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:57.274365    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:57.593909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:57.593872    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:57.744514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:57.743955    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:58.109787 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:58.109755    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:58.988661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:58.988626    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:42:59.674182 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:42:59.671910    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:00.700040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:00.699965    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:01.574120 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:01.574078    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:02.313982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:02.313916    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:03.102813 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:03.102781    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:04.064440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:04.064357    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:04.887028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:04.886987    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:05.738535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:05.738443    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:06.507461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:06.507383    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:07.375296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:07.375258    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:08.095989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:08.095954    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:08.976712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:08.976682    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:09.865921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:09.865878    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:10.024639 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:43:10.024598    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:43:10.737769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:10.737642    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:11.734080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:11.734045    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:12.599355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:12.599322    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:13.363773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:13.363735    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:13.825166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:13.825133    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:14.350544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:14.350442    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:14.989734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:14.989695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:15.502601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.502563    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"]
Nov 01 22:43:15.502805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.502650    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"]
Nov 01 22:43:15.502904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.502858    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" containerID="cri-o://e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f" gracePeriod=30
Nov 01 22:43:15.502978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.502893    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" containerID="cri-o://9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6" gracePeriod=30
Nov 01 22:43:15.507354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.507325    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"]
Nov 01 22:43:15.507868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.507588    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy" containerID="cri-o://ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6" gracePeriod=30
Nov 01 22:43:15.952403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:15.952368    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:16.746529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:16.746477    5057 patch_prober.go:29] interesting pod/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:43:16.746934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:16.746572    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:43:17.291552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:17.289846    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:18.292515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:18.287569    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:19.090797 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:19.090082    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:19.256303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:19.256264    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:43:19.256537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:19.256325    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:43:20.094344 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.094301    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:20.847780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.847699    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:43:20.891342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.891301    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"]
Nov 01 22:43:20.892315 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.892289    5057 cpu_manager.go:414] "containerMap: removing container" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy"
Nov 01 22:43:20.892315 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.892316    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy"
Nov 01 22:43:20.892596 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.892578    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerName="istio-proxy"
Nov 01 22:43:20.898081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.898059    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:20.902366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.902342    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-ee8d1a-openshift-default-dockercfg-hsk87\""
Nov 01 22:43:20.903458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.903431    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.904010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.903987    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904097    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904189    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904221    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904270    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904346    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904371    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904556    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904784    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.904900    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"]
Nov 01 22:43:20.905248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.905077    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:20.905806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.905418    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.905454    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert\") pod \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\" (UID: \"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39\") "
Nov 01 22:43:20.905806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.905749    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:20.906617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.906585    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:20.906712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.906616    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:20.906712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.906632    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:20.906712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.906645    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:20.906889 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.906867    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:43:20.907811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.907771    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "kube-api-access-49rll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:20.910551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.909131    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:20.926995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.926945    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:43:20.928335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.928305    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token" pod "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" (UID: "4d6bb450-46b0-4101-a8c5-5ba6da4e9c39"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:20.931415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.931391    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:43:20.970771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.970457    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:43:20.996337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.996297    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"]
Nov 01 22:43:20.998903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.998877    5057 cpu_manager.go:414] "containerMap: removing container" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy"
Nov 01 22:43:20.999114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.999100    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy"
Nov 01 22:43:20.999208 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.999199    5057 cpu_manager.go:414] "containerMap: removing container" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy"
Nov 01 22:43:20.999284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.999274    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy"
Nov 01 22:43:20.999592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.999561    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerName="istio-proxy"
Nov 01 22:43:20.999700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:20.999689    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="56baaa56-760a-4802-90ed-116711c2304d" containerName="istio-proxy"
Nov 01 22:43:21.006471 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.006452    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.007264    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.008477    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.008609    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.008653    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.008700    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.008923    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.008968    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.009012    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.009539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.009077    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token\") pod \"56baaa56-760a-4802-90ed-116711c2304d\" (UID: \"56baaa56-760a-4802-90ed-116711c2304d\") "
Nov 01 22:43:21.010512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.010140    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.010512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.010264    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.010512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.010299    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.010512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.010449    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.010891    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbgv6\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.010955    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011006    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011094    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011133    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011299    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011321    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011337    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011352    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.011366    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49rll\" (UniqueName: \"kubernetes.io/projected/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39-kube-api-access-49rll\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.011518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.007566    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.014899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.014878    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.015221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.015204    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.015618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.015599    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:43:21.015741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.015727    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"]
Nov 01 22:43:21.016091 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.016066    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.025700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.025652    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a9c08e-openshift-default-dockercfg-kprwc\""
Nov 01 22:43:21.027901 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.027864    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:21.030810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.030782    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:43:21.030957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.030903    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "kube-api-access-6mmvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:21.031391 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.031356    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy" pod "56baaa56-760a-4802-90ed-116711c2304d" (UID: "56baaa56-760a-4802-90ed-116711c2304d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.039904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.039739    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"]
Nov 01 22:43:21.047795 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.047273    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.054012 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.053991    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"]
Nov 01 22:43:21.112397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.112362    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.112844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.112480    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.112844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.112556    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.112844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.112709    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.112844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.112729    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.113067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.112868    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.113067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113000    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.113200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113095    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.113200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113135    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.113200 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113170    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113213    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert\") pod \"0be1d300-18b4-47c5-b74d-05bce19f3411\" (UID: \"0be1d300-18b4-47c5-b74d-05bce19f3411\") "
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113616    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113758    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113800    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113905    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113938    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113950    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.113972    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114124    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114162    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114183    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114229    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114299    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114324    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114353    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114564    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114697    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114723    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114763    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95hk\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114798    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4t9dw\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114830    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbgv6\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114855    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114879    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114893    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114942    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.114996    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115077    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115114    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115154    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115184    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115189    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115253    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115382    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115409    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115459    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115671    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115697    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115715    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mmvf\" (UniqueName: \"kubernetes.io/projected/56baaa56-760a-4802-90ed-116711c2304d-kube-api-access-6mmvf\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115731    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115744    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0be1d300-18b4-47c5-b74d-05bce19f3411-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115757    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56baaa56-760a-4802-90ed-116711c2304d-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115770    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115783    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56baaa56-760a-4802-90ed-116711c2304d-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115796    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115813    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115826    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115842    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115856    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115869    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115877    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.115882    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56baaa56-760a-4802-90ed-116711c2304d-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.116104    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.116128    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "kube-api-access-fwjcd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:21.116302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.116315    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.127263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.117897    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token" pod "0be1d300-18b4-47c5-b74d-05bce19f3411" (UID: "0be1d300-18b4-47c5-b74d-05bce19f3411"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:21.127263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.118127    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.127263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.119888    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.175739 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.153015    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.187784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.187710    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbgv6\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.217926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.217886    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.217943    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.217982    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218007    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218055    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95hk\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218086    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4t9dw\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218118    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218166    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218286    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218327    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218349    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218380    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218424    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218518    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218620    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218728    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.218770 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218757    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.218998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.218779    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.219484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.219175    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.219484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.219385    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.219484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.219476    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.220344    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.220416    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.221843    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.221867    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0be1d300-18b4-47c5-b74d-05bce19f3411-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.221882    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwjcd\" (UniqueName: \"kubernetes.io/projected/0be1d300-18b4-47c5-b74d-05bce19f3411-kube-api-access-fwjcd\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.222553    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.222625    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.222841    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.223095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.223024    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.223694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.223319    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.223694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.223373    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.226241 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.226215    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.226764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.226734    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.227553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.227485    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.230423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.229283    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.230423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.230364    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.231437 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.231414    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4t9dw\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw\") pod \"e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") " pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.231577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.231457    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95hk\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk\") pod \"e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") " pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.254319 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.253906    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:21.340545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.340216    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:21.370810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.367486    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:21.599302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.599206    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"]
Nov 01 22:43:21.618282 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.618229    5057 generic.go:356] "Generic (PLEG): container finished" podID="0be1d300-18b4-47c5-b74d-05bce19f3411" containerID="ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6" exitCode=0
Nov 01 22:43:21.619585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.618683    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" event={"ID":"0be1d300-18b4-47c5-b74d-05bce19f3411","Type":"ContainerDied","Data":"ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6"}
Nov 01 22:43:21.619585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.618744    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr" event={"ID":"0be1d300-18b4-47c5-b74d-05bce19f3411","Type":"ContainerDied","Data":"149b5ae805a3f6ee3226c34597f1aa31d34ff8b5b850a028a3693147455c27c5"}
Nov 01 22:43:21.619585 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:43:21.618832    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7792407c_331d_46d3_b2a3_7b7d2c48d5fa.slice/crio-2a7c77ec4de98d166a8d5b567607b8b96e845a744d3f9ed904668f5747e221c0 WatchSource:0}: Error finding container 2a7c77ec4de98d166a8d5b567607b8b96e845a744d3f9ed904668f5747e221c0: Status 404 returned error can't find the container with id 2a7c77ec4de98d166a8d5b567607b8b96e845a744d3f9ed904668f5747e221c0
Nov 01 22:43:21.619585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.618948    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"
Nov 01 22:43:21.620022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.619844    5057 scope.go:122] "RemoveContainer" containerID="ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6"
Nov 01 22:43:21.637044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.636760    5057 generic.go:356] "Generic (PLEG): container finished" podID="56baaa56-760a-4802-90ed-116711c2304d" containerID="9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6" exitCode=0
Nov 01 22:43:21.637044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.636992    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"
Nov 01 22:43:21.637187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.637135    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:21.637247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.637227    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:21.637296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.637271    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:21.637688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.637590    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" event={"ID":"56baaa56-760a-4802-90ed-116711c2304d","Type":"ContainerDied","Data":"9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6"}
Nov 01 22:43:21.637688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.637655    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq" event={"ID":"56baaa56-760a-4802-90ed-116711c2304d","Type":"ContainerDied","Data":"2ca842d240e6b1d566f3cc23fce4802fc251d1c172e980d792d01cce0ad9f218"}
Nov 01 22:43:21.645520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.641598    5057 generic.go:356] "Generic (PLEG): container finished" podID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" containerID="e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f" exitCode=0
Nov 01 22:43:21.645520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.642130    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"
Nov 01 22:43:21.645520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.643002    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" event={"ID":"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39","Type":"ContainerDied","Data":"e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f"}
Nov 01 22:43:21.645520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.643052    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg" event={"ID":"4d6bb450-46b0-4101-a8c5-5ba6da4e9c39","Type":"ContainerDied","Data":"1f305818298dc09f3b459a16e2ad29bccbafe4b12094041b4717d43a74bb3d6a"}
Nov 01 22:43:21.680052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.680029    5057 scope.go:122] "RemoveContainer" containerID="ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6"
Nov 01 22:43:21.680629 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:43:21.680602    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6\": container with ID starting with ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6 not found: ID does not exist" containerID="ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6"
Nov 01 22:43:21.680792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.680767    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6"} err="failed to get container status \"ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6\": rpc error: code = NotFound desc = could not find container \"ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6\": container with ID starting with ab5c04bb8983ea5778da0034cff79124991cabab8921a18fb98489c8a7e414d6 not found: ID does not exist"
Nov 01 22:43:21.681193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.680892    5057 scope.go:122] "RemoveContainer" containerID="9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6"
Nov 01 22:43:21.699555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.699527    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"]
Nov 01 22:43:21.710110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.710075    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-pwhwr"]
Nov 01 22:43:21.714677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.714651    5057 scope.go:122] "RemoveContainer" containerID="9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6"
Nov 01 22:43:21.715179 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:43:21.715120    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6\": container with ID starting with 9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6 not found: ID does not exist" containerID="9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6"
Nov 01 22:43:21.715335 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.715309    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6"} err="failed to get container status \"9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6\": rpc error: code = NotFound desc = could not find container \"9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6\": container with ID starting with 9d16afa1fdd6fd811d1a8fce3791275217363e33a7c1554ec99bb7cd708922f6 not found: ID does not exist"
Nov 01 22:43:21.715750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.715733    5057 scope.go:122] "RemoveContainer" containerID="e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f"
Nov 01 22:43:21.720523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.720482    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"]
Nov 01 22:43:21.724357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.724325    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:21.724461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.724404    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:21.724461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.724453    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:21.763289 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.763256    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"]
Nov 01 22:43:21.772674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.772647    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-28nfg"]
Nov 01 22:43:21.775835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.773318    5057 scope.go:122] "RemoveContainer" containerID="e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f"
Nov 01 22:43:21.776080 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:43:21.776055    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f\": container with ID starting with e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f not found: ID does not exist" containerID="e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f"
Nov 01 22:43:21.776180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.776122    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f"} err="failed to get container status \"e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f\": rpc error: code = NotFound desc = could not find container \"e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f\": container with ID starting with e9856a9205e491758ccc1cd657944b6cf51115f7a062e220ce683ce3d7e5301f not found: ID does not exist"
Nov 01 22:43:21.794208 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.794182    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"]
Nov 01 22:43:21.802201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.802177    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-q6ldq"]
Nov 01 22:43:21.882098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.881969    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:21.997088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:21.996777    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"]
Nov 01 22:43:21.997199 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:43:21.997107    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaead9d7_2777_4ed1_943d_382abf76fe36.slice/crio-88180b71ffeae2b96a8a478ecdb5ede3b73de1ca89d4b8aa1f26606f78995b5a WatchSource:0}: Error finding container 88180b71ffeae2b96a8a478ecdb5ede3b73de1ca89d4b8aa1f26606f78995b5a: Status 404 returned error can't find the container with id 88180b71ffeae2b96a8a478ecdb5ede3b73de1ca89d4b8aa1f26606f78995b5a
Nov 01 22:43:22.001584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.001542    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:22.001680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.001631    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:22.001680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.001671    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:43:22.656796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.656757    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" event={"ID":"daead9d7-2777-4ed1-943d-382abf76fe36","Type":"ContainerStarted","Data":"a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3"}
Nov 01 22:43:22.657236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.657207    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" event={"ID":"daead9d7-2777-4ed1-943d-382abf76fe36","Type":"ContainerStarted","Data":"88180b71ffeae2b96a8a478ecdb5ede3b73de1ca89d4b8aa1f26606f78995b5a"}
Nov 01 22:43:22.659436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.659403    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" event={"ID":"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7","Type":"ContainerStarted","Data":"6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"}
Nov 01 22:43:22.659559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.659456    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" event={"ID":"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7","Type":"ContainerStarted","Data":"11f404fd0ce23f45343fe6946ed7289068c5c6b52f0f487453e4d687f0f43b7e"}
Nov 01 22:43:22.661960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.661942    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" event={"ID":"7792407c-331d-46d3-b2a3-7b7d2c48d5fa","Type":"ContainerStarted","Data":"0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7"}
Nov 01 22:43:22.662061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.661977    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" event={"ID":"7792407c-331d-46d3-b2a3-7b7d2c48d5fa","Type":"ContainerStarted","Data":"2a7c77ec4de98d166a8d5b567607b8b96e845a744d3f9ed904668f5747e221c0"}
Nov 01 22:43:22.699131 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.699100    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:22.699278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.699147    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podStartSLOduration=48.699134052 podStartE2EDuration="48.699134052s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:42:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:43:22.692972572 +0000 UTC m=+1259.434668196" watchObservedRunningTime="2026-11-01 22:43:22.699134052 +0000 UTC m=+1259.440829671"
Nov 01 22:43:22.719968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.719905    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podStartSLOduration=3.719887217 podStartE2EDuration="3.719887217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:43:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:43:22.713950495 +0000 UTC m=+1259.455646113" watchObservedRunningTime="2026-11-01 22:43:22.719887217 +0000 UTC m=+1259.461582837"
Nov 01 22:43:22.741714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.741668    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podStartSLOduration=25.741652465 podStartE2EDuration="25.741652465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:42:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:43:22.737438204 +0000 UTC m=+1259.479133823" watchObservedRunningTime="2026-11-01 22:43:22.741652465 +0000 UTC m=+1259.483348086"
Nov 01 22:43:22.826412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.826376    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"]
Nov 01 22:43:22.826618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:22.826576    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy" containerID="cri-o://2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32" gracePeriod=30
Nov 01 22:43:23.254467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.254428    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:23.256244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.256214    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:23.256412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.256265    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:23.341007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.340975    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:23.343207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.343176    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:43:23.343541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.343224    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:43:23.369915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.369886    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:23.371532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.371502    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:23.371659 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.371556    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:23.399731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.399703    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0be1d300-18b4-47c5-b74d-05bce19f3411" path="/var/lib/kubelet/pods/0be1d300-18b4-47c5-b74d-05bce19f3411/volumes"
Nov 01 22:43:23.400399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.400375    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d6bb450-46b0-4101-a8c5-5ba6da4e9c39" path="/var/lib/kubelet/pods/4d6bb450-46b0-4101-a8c5-5ba6da4e9c39/volumes"
Nov 01 22:43:23.401051 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.401033    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="56baaa56-760a-4802-90ed-116711c2304d" path="/var/lib/kubelet/pods/56baaa56-760a-4802-90ed-116711c2304d/volumes"
Nov 01 22:43:23.575067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:23.574972    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:24.255411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.255377    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:24.255956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.255437    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:24.340692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.340653    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:43:24.340872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.340720    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:43:24.369199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.369163    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:24.369386 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.369219    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:24.566245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:24.566056    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:25.179600 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.178654    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:25.258011 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.257969    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:25.258649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.258614    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:25.371550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.369875    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:25.371550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.369934    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:25.383583 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.376779    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:25.393403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.393374    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:25.704209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.704180    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:25.705969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.705616    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:43:25.993918 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:25.993655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:26.274645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:26.255547    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:26.274645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:26.255618    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:26.381551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:26.376059    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:26.381551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:26.376147    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:27.263519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:27.259629    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:27.263519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:27.259700    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:27.376515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:27.370895    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:27.376515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:27.370979    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:27.595248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:27.595146    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:28.255651 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.255614    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:28.255921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.255893    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:28.396898 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.396850    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:28.397417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.396932    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:28.615267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.615240    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:43:28.813702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.813609    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.813920 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.813899    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:28.814024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.813645    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.813982    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814012    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:28.814134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814093    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814160    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814271    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814289    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814308    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814423    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy\") pod \"031e1687-4ecd-4efb-b046-da95c18d4cb8\" (UID: \"031e1687-4ecd-4efb-b046-da95c18d4cb8\") "
Nov 01 22:43:28.814636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814601    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:28.814952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.814921    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:43:28.815173 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.815118    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:28.815535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.815514    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.815673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.815658    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.815762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.815750    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/031e1687-4ecd-4efb-b046-da95c18d4cb8-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.815850 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.815838    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.815934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.815922    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.817351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.817329    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:43:28.873810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.873724    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "kube-api-access-9dt67". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:28.893910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.892360    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:43:28.896341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.896161    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token" pod "031e1687-4ecd-4efb-b046-da95c18d4cb8" (UID: "031e1687-4ecd-4efb-b046-da95c18d4cb8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:43:28.898571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.897653    5057 generic.go:356] "Generic (PLEG): container finished" podID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerID="2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32" exitCode=0
Nov 01 22:43:28.898571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.897847    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" event={"ID":"031e1687-4ecd-4efb-b046-da95c18d4cb8","Type":"ContainerDied","Data":"2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32"}
Nov 01 22:43:28.898571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.897884    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w" event={"ID":"031e1687-4ecd-4efb-b046-da95c18d4cb8","Type":"ContainerDied","Data":"42fc1daa88c8568e816bc46897c1f2bf444ed5ce164a38b3100d00d2a28a67fb"}
Nov 01 22:43:28.898571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.897906    5057 scope.go:122] "RemoveContainer" containerID="2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32"
Nov 01 22:43:28.898571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.898024    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.918540    5057 scope.go:122] "RemoveContainer" containerID="2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32"
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:43:28.919140    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32\": container with ID starting with 2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32 not found: ID does not exist" containerID="2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32"
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.919180    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32"} err="failed to get container status \"2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32\": rpc error: code = NotFound desc = could not find container \"2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32\": container with ID starting with 2c7099d4163794cec7837e5c9ff6612e456213f3acf0a89170163deaf03fae32 not found: ID does not exist"
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.920537    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.920562    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dt67\" (UniqueName: \"kubernetes.io/projected/031e1687-4ecd-4efb-b046-da95c18d4cb8-kube-api-access-9dt67\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.920580    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.922762 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.920593    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/031e1687-4ecd-4efb-b046-da95c18d4cb8-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:43:28.985520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.985090    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"]
Nov 01 22:43:28.989306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:28.989279    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-dfx5w"]
Nov 01 22:43:29.039304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.039202    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:29.209201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.203699    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:29.261045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.257470    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:29.261045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.257550    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:29.379590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.370747    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:29.379590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.370803    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:29.400775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:29.400743    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" path="/var/lib/kubelet/pods/031e1687-4ecd-4efb-b046-da95c18d4cb8/volumes"
Nov 01 22:43:30.262521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:30.257601    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:30.262521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:30.257706    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body=
Nov 01 22:43:30.262521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:30.257746    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused"
Nov 01 22:43:30.372086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:30.370885    5057 patch_prober.go:29] interesting pod/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused" start-of-body=
Nov 01 22:43:30.372086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:30.370955    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.106:15021/healthz/ready\": dial tcp 10.132.0.106:15021: connect: connection refused"
Nov 01 22:43:31.173523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.173182    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:31.262008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.261970    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:31.262448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.262422    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:31.263218 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.263194    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:43:31.381303 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.381274    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:31.381720 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.381661    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:31.382688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.382665    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:43:31.580406 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:31.580344    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:32.125254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:32.125073    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:33.386961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:33.386915    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:34.615804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:34.615760    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:35.113821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:35.113780    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:35.257838 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:35.257805    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:35.405284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:35.405202    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:36.272433 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:36.272398    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:36.835340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:36.835295    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:37.305672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:37.305632    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:37.905428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:37.905381    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:38.033910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:38.033873    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:38.679100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:38.679065    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:39.332775 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:39.332720    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:39.467590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:39.467550    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:39.605580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:39.605445    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:40.026561 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:43:40.026518    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:43:40.078981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:40.078947    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:43:40.623998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:40.623052    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:41.633108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:41.633068    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:41.824363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:41.824321    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:41.965486 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:41.965440    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:42.592525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:42.591866    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:43.205044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:43.205005    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:43.984275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:43.984241    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:44.792893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:44.792854    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:45.702993 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.702943    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"]
Nov 01 22:43:45.704307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.704279    5057 cpu_manager.go:414] "containerMap: removing container" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy"
Nov 01 22:43:45.704452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.704440    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy"
Nov 01 22:43:45.704832 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.704816    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="031e1687-4ecd-4efb-b046-da95c18d4cb8" containerName="istio-proxy"
Nov 01 22:43:45.719530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.709736    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.736906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.736880    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-a9c08e-serving-cert\""
Nov 01 22:43:45.741484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.738030    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"]
Nov 01 22:43:45.800880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.800841    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:45.842447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.842409    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.842690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.842568    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.842894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.842869    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkt4p\" (UniqueName: \"kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.843251 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.843227    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.974386 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.974300    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:45.981223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.976907    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.986873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.986825    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkt4p\" (UniqueName: \"kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.987455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.987417    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.987930 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.987910    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.995189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.985696    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.995189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.991050    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:45.997225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:45.997124    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:46.012830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:46.012801    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkt4p\" (UniqueName: \"kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p\") pod \"maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:46.055881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:46.055843    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:46.109776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:46.109751    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:46.385237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:46.385194    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"]
Nov 01 22:43:46.989473 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:46.988822    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:47.101616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:47.101526    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4" event={"ID":"96be7af1-f3bf-4aa2-82c7-24d84f474120","Type":"ContainerStarted","Data":"3a8a95f27fdb8de082954b4252c7d6337517be24e67f4cd05271b2d0615dbbb4"}
Nov 01 22:43:47.662934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:47.662887    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:47.777781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:47.777749    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:48.124038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:48.124001    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:48.126254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:48.126220    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4" event={"ID":"96be7af1-f3bf-4aa2-82c7-24d84f474120","Type":"ContainerStarted","Data":"e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae"}
Nov 01 22:43:48.129655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:48.127165    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:48.157955 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:48.157884    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4" podStartSLOduration=2.755609286 podStartE2EDuration="3.157868065s" totalImagesPullingTime="402.258779ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:43:48.151989556 +0000 UTC m=+1284.893685176" watchObservedRunningTime="2026-11-01 22:43:48.157868065 +0000 UTC m=+1284.899563686"
Nov 01 22:43:48.254231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:48.254194    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:49.248517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:49.239960    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:49.400346 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:49.400306    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:49.552400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:49.552250    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:50.299456 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:50.299428    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:50.433675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:50.433423    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:51.471020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:51.470983    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:52.302603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:52.302563    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:52.693710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:52.693580    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:52.857134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:52.857095    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:53.303381 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:53.303348    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:53.744560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:53.744522    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:54.147755 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.147673    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:43:54.196065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.196029    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:54.301293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.301258    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"]
Nov 01 22:43:54.309967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.309935    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:54.317001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.316975    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"]
Nov 01 22:43:54.353551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.353517    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:54.380867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.380768    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:54.381382 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.381084    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2sb\" (UniqueName: \"kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb\") pod \"payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:54.393275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.392895    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-8f2555\""
Nov 01 22:43:54.393275 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.393085    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-8f2555-dockercfg-jmd6w\""
Nov 01 22:43:54.593253 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.592188    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:54.628580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.625409    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2sb\" (UniqueName: \"kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb\") pod \"payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:54.628580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.625779    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:54.639934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.629694    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:54.791306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:54.791269    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2sb\" (UniqueName: \"kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb\") pod \"payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:55.036186 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:55.036157    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:55.329104 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:55.329074    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"]
Nov 01 22:43:55.330405 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:43:55.330372    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6254f311_ea06_458c_abf6_b057560c94ba.slice/crio-ee875c7dd44f569a22ff0e80b27a4eee0b0a3d9ee5afd50f2ccaeee45951bb1b WatchSource:0}: Error finding container ee875c7dd44f569a22ff0e80b27a4eee0b0a3d9ee5afd50f2ccaeee45951bb1b: Status 404 returned error can't find the container with id ee875c7dd44f569a22ff0e80b27a4eee0b0a3d9ee5afd50f2ccaeee45951bb1b
Nov 01 22:43:55.460058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:55.460023    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:56.205015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:56.199931    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:56.241281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:56.241241    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4" event={"ID":"6254f311-ea06-458c-abf6-b057560c94ba","Type":"ContainerStarted","Data":"ee875c7dd44f569a22ff0e80b27a4eee0b0a3d9ee5afd50f2ccaeee45951bb1b"}
Nov 01 22:43:56.351814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:56.351775    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:57.333412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:57.333334    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:57.998531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:57.998475    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:58.707076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:58.707044    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:59.272937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:59.272905    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4" event={"ID":"6254f311-ea06-458c-abf6-b057560c94ba","Type":"ContainerStarted","Data":"2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870"}
Nov 01 22:43:59.273112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:59.273054    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:43:59.293640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:59.293589    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4" podStartSLOduration=1.823118563 podStartE2EDuration="5.293577161s" totalImagesPullingTime="3.470458598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:43:59.289993589 +0000 UTC m=+1296.031689219" watchObservedRunningTime="2026-11-01 22:43:59.293577161 +0000 UTC m=+1296.035272779"
Nov 01 22:43:59.463033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:59.462995    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:59.810545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:59.809418    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:43:59.936065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:43:59.936034    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:00.398512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:00.397652    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:01.051092 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:01.051054    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:01.624176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:01.624133    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:02.058846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:02.058806    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:02.682635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:02.682597    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:03.374778 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:03.374745    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:04.132487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:04.132443    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:04.631129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:04.631089    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"]
Nov 01 22:44:04.631674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:04.631330    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw" podUID="b1d559e1-a742-4dde-b035-ba712f750510" containerName="maas-api" containerID="cri-o://5ee26d67034b03523c455c2d60ad35bfb1f47465071d4e77ce01c7daecfe1095" gracePeriod=30
Nov 01 22:44:04.724185 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:04.724021    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:04.911858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:04.911762    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:05.106082 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.106053    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:05.356428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.356390    5057 generic.go:356] "Generic (PLEG): container finished" podID="b1d559e1-a742-4dde-b035-ba712f750510" containerID="5ee26d67034b03523c455c2d60ad35bfb1f47465071d4e77ce01c7daecfe1095" exitCode=0
Nov 01 22:44:05.356628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.356561    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw" event={"ID":"b1d559e1-a742-4dde-b035-ba712f750510","Type":"ContainerDied","Data":"5ee26d67034b03523c455c2d60ad35bfb1f47465071d4e77ce01c7daecfe1095"}
Nov 01 22:44:05.402752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.402721    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:44:05.478485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.478437    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls\") pod \"b1d559e1-a742-4dde-b035-ba712f750510\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") "
Nov 01 22:44:05.494062 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.494013    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls" pod "b1d559e1-a742-4dde-b035-ba712f750510" (UID: "b1d559e1-a742-4dde-b035-ba712f750510"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:44:05.562723 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.562621    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:05.582684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.582563    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs\") pod \"b1d559e1-a742-4dde-b035-ba712f750510\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") "
Nov 01 22:44:05.582684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.582643    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t\" (UniqueName: \"kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t\") pod \"b1d559e1-a742-4dde-b035-ba712f750510\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") "
Nov 01 22:44:05.583201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.583113    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca\") pod \"b1d559e1-a742-4dde-b035-ba712f750510\" (UID: \"b1d559e1-a742-4dde-b035-ba712f750510\") "
Nov 01 22:44:05.585321 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.584940    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:05.593896 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.592296    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs" pod "b1d559e1-a742-4dde-b035-ba712f750510" (UID: "b1d559e1-a742-4dde-b035-ba712f750510"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:44:05.594277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.594199    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca" pod "b1d559e1-a742-4dde-b035-ba712f750510" (UID: "b1d559e1-a742-4dde-b035-ba712f750510"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:05.623898 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.623837    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t" pod "b1d559e1-a742-4dde-b035-ba712f750510" (UID: "b1d559e1-a742-4dde-b035-ba712f750510"). InnerVolumeSpecName "kube-api-access-t4f6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:05.687331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.686985    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b1d559e1-a742-4dde-b035-ba712f750510-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:05.687331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.687021    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4f6t\" (UniqueName: \"kubernetes.io/projected/b1d559e1-a742-4dde-b035-ba712f750510-kube-api-access-t4f6t\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:05.687331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.687031    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b1d559e1-a742-4dde-b035-ba712f750510-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:05.903730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:05.903652    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:06.464118 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.464078    5057 generic.go:356] "Generic (PLEG): container finished" podID="5214431e-5fc7-4ec2-af7c-e32a789c7d0f" containerID="5ff95fa5ed6fbb4c45cd6b93d3f0e5cc74a73e2e720f8387b4465ec58ddf6737" exitCode=0
Nov 01 22:44:06.464584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.464558    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc" event={"ID":"5214431e-5fc7-4ec2-af7c-e32a789c7d0f","Type":"ContainerDied","Data":"5ff95fa5ed6fbb4c45cd6b93d3f0e5cc74a73e2e720f8387b4465ec58ddf6737"}
Nov 01 22:44:06.471577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.471452    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"
Nov 01 22:44:06.471841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.471474    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw" event={"ID":"b1d559e1-a742-4dde-b035-ba712f750510","Type":"ContainerDied","Data":"7a7e93ca0001bc79f71f466240e6c708566f1426465662ee2442466e296f91d9"}
Nov 01 22:44:06.471931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.471884    5057 scope.go:122] "RemoveContainer" containerID="5ee26d67034b03523c455c2d60ad35bfb1f47465071d4e77ce01c7daecfe1095"
Nov 01 22:44:06.543914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.539859    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"]
Nov 01 22:44:06.559573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.558315    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a1d4be-7bc84578d6-4hpvw"]
Nov 01 22:44:06.735366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:06.735273    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:07.368418 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.368381    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:07.400277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.400243    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b1d559e1-a742-4dde-b035-ba712f750510" path="/var/lib/kubelet/pods/b1d559e1-a742-4dde-b035-ba712f750510/volumes"
Nov 01 22:44:07.708548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.708517    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:44:07.735651 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.735620    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle\") pod \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") "
Nov 01 22:44:07.736175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.735681    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95\" (UniqueName: \"kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95\") pod \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") "
Nov 01 22:44:07.736175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.735729    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle\") pod \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") "
Nov 01 22:44:07.736175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.735934    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path\") pod \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\" (UID: \"5214431e-5fc7-4ec2-af7c-e32a789c7d0f\") "
Nov 01 22:44:07.736175 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.736122    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle" pod "5214431e-5fc7-4ec2-af7c-e32a789c7d0f" (UID: "5214431e-5fc7-4ec2-af7c-e32a789c7d0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:07.736367 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.736203    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle" pod "5214431e-5fc7-4ec2-af7c-e32a789c7d0f" (UID: "5214431e-5fc7-4ec2-af7c-e32a789c7d0f"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:07.736407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.736374    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path" pod "5214431e-5fc7-4ec2-af7c-e32a789c7d0f" (UID: "5214431e-5fc7-4ec2-af7c-e32a789c7d0f"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:07.736870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.736847    5057 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-trusted-ca-bundle\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:07.736937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.736875    5057 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-service-ca-bundle\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:07.736937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.736891    5057 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-archives-path\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:07.738243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.738217    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95" pod "5214431e-5fc7-4ec2-af7c-e32a789c7d0f" (UID: "5214431e-5fc7-4ec2-af7c-e32a789c7d0f"). InnerVolumeSpecName "kube-api-access-6nq95". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:07.841853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.841809    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nq95\" (UniqueName: \"kubernetes.io/projected/5214431e-5fc7-4ec2-af7c-e32a789c7d0f-kube-api-access-6nq95\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:07.846244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:07.845630    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:08.490229 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:08.490195    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc" event={"ID":"5214431e-5fc7-4ec2-af7c-e32a789c7d0f","Type":"ContainerDied","Data":"617f6341e36e1054c99ff593d8f157e28a7ec0dcbe23a2092328401c4f2ccb45"}
Nov 01 22:44:08.490404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:08.490247    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="617f6341e36e1054c99ff593d8f157e28a7ec0dcbe23a2092328401c4f2ccb45"
Nov 01 22:44:08.490404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:08.490204    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-kbdg8-jm4bc"
Nov 01 22:44:08.731290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:08.731240    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:08.934793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:08.933598    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:09.047815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:09.047783    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:09.562600 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:09.562560    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:10.024785 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:10.024743    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:44:10.084512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.083893    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"]
Nov 01 22:44:10.084512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.084117    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" podUID="ca20de96-6023-4e74-b8f1-d65d2123e913" containerName="maas-api" containerID="cri-o://3287ec2727966e43f5361bd0a2b71cdd9b63af963717ec3ff38d959095c66478" gracePeriod=30
Nov 01 22:44:10.167314 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.166265    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:10.406576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.391703    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:44:10.490025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.487986    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:10.804206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.804089    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:10.811601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.811560    5057 generic.go:356] "Generic (PLEG): container finished" podID="ca20de96-6023-4e74-b8f1-d65d2123e913" containerID="3287ec2727966e43f5361bd0a2b71cdd9b63af963717ec3ff38d959095c66478" exitCode=0
Nov 01 22:44:10.811790 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.811772    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" event={"ID":"ca20de96-6023-4e74-b8f1-d65d2123e913","Type":"ContainerDied","Data":"3287ec2727966e43f5361bd0a2b71cdd9b63af963717ec3ff38d959095c66478"}
Nov 01 22:44:10.895086 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:10.894893    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:44:11.026332 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.026297    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca\") pod \"ca20de96-6023-4e74-b8f1-d65d2123e913\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") "
Nov 01 22:44:11.026868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.026385    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs\") pod \"ca20de96-6023-4e74-b8f1-d65d2123e913\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") "
Nov 01 22:44:11.026868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.026417    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5\" (UniqueName: \"kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5\") pod \"ca20de96-6023-4e74-b8f1-d65d2123e913\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") "
Nov 01 22:44:11.026868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.026447    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\") pod \"ca20de96-6023-4e74-b8f1-d65d2123e913\" (UID: \"ca20de96-6023-4e74-b8f1-d65d2123e913\") "
Nov 01 22:44:11.026868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.026747    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca" pod "ca20de96-6023-4e74-b8f1-d65d2123e913" (UID: "ca20de96-6023-4e74-b8f1-d65d2123e913"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:11.030239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.028302    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca20de96-6023-4e74-b8f1-d65d2123e913-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:11.030239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.029539    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs" pod "ca20de96-6023-4e74-b8f1-d65d2123e913" (UID: "ca20de96-6023-4e74-b8f1-d65d2123e913"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:44:11.030239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.029920    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls" pod "ca20de96-6023-4e74-b8f1-d65d2123e913" (UID: "ca20de96-6023-4e74-b8f1-d65d2123e913"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:44:11.030719 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.030696    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5" pod "ca20de96-6023-4e74-b8f1-d65d2123e913" (UID: "ca20de96-6023-4e74-b8f1-d65d2123e913"). InnerVolumeSpecName "kube-api-access-6q8n5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:11.097986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.093360    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:11.131220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.131191    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:11.131220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.131218    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q8n5\" (UniqueName: \"kubernetes.io/projected/ca20de96-6023-4e74-b8f1-d65d2123e913-kube-api-access-6q8n5\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:11.131446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.131247    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca20de96-6023-4e74-b8f1-d65d2123e913-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:11.268372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.268330    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:11.464085 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.464051    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:11.620322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.620280    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:11.828222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.828184    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr" event={"ID":"ca20de96-6023-4e74-b8f1-d65d2123e913","Type":"ContainerDied","Data":"9ce840c2f0c5ef5e4841298b568824d3624adfc1c2b8547a88161d47ab962c63"}
Nov 01 22:44:11.828413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.828261    5057 scope.go:122] "RemoveContainer" containerID="3287ec2727966e43f5361bd0a2b71cdd9b63af963717ec3ff38d959095c66478"
Nov 01 22:44:11.828413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.828409    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"
Nov 01 22:44:11.879026 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.878990    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"]
Nov 01 22:44:11.910410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.910372    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-ee8d1a-75f599cfc7-pjmmr"]
Nov 01 22:44:11.950061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.950020    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:11.966759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.965948    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"]
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967427    5057 cpu_manager.go:414] "containerMap: removing container" podUID="ca20de96-6023-4e74-b8f1-d65d2123e913" containerName="maas-api"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967455    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca20de96-6023-4e74-b8f1-d65d2123e913" containerName="maas-api"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967468    5057 cpu_manager.go:414] "containerMap: removing container" podUID="5214431e-5fc7-4ec2-af7c-e32a789c7d0f" containerName="insights-gathering"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967474    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5214431e-5fc7-4ec2-af7c-e32a789c7d0f" containerName="insights-gathering"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967539    5057 cpu_manager.go:414] "containerMap: removing container" podUID="b1d559e1-a742-4dde-b035-ba712f750510" containerName="maas-api"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967547    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b1d559e1-a742-4dde-b035-ba712f750510" containerName="maas-api"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967753    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="b1d559e1-a742-4dde-b035-ba712f750510" containerName="maas-api"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967779    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca20de96-6023-4e74-b8f1-d65d2123e913" containerName="maas-api"
Nov 01 22:44:11.968015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.967789    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="5214431e-5fc7-4ec2-af7c-e32a789c7d0f" containerName="insights-gathering"
Nov 01 22:44:11.977536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.974828    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:11.988656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.988620    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8f2555-serving-cert\""
Nov 01 22:44:11.998205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:11.998164    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"]
Nov 01 22:44:12.051539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.051452    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.052258 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.052217    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.052576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.052559    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.052978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.052954    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9rfd\" (UniqueName: \"kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.096609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.093558    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:12.178540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.178487    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9rfd\" (UniqueName: \"kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.178952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.178927    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.179195 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.179176    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.192921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.192886    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.194169 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:12.179362    5057 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-serving-cert: secret "maas-api-e2e-api-b-8f2555-serving-cert" not found
Nov 01 22:44:12.194407 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:12.194390    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls podName:5eb7ec26-3c0c-414c-995c-503eeb590770 nodeName:}" failed. No retries permitted until 2026-11-01 22:44:12.694367121 +0000 UTC m=+1309.436062730 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls") pod "maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" (UID: "5eb7ec26-3c0c-414c-995c-503eeb590770") : secret "maas-api-e2e-api-b-8f2555-serving-cert" not found
Nov 01 22:44:12.194739 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.193839    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.208517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.207855    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9rfd\" (UniqueName: \"kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.245423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.245392    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.259528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.247138    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:12.442052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.438699    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"]
Nov 01 22:44:12.642714 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.642271    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:12.747255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.747214    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.750633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.750601    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\") pod \"maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:12.843997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.843959    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:12.907064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:12.907031    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:13.274369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:13.274219    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"]
Nov 01 22:44:13.278636 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:44:13.278590    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eb7ec26_3c0c_414c_995c_503eeb590770.slice/crio-c13767ed15f59e306219fd7caaf739ddb329e744a162ab23fde41b3101e85291 WatchSource:0}: Error finding container c13767ed15f59e306219fd7caaf739ddb329e744a162ab23fde41b3101e85291: Status 404 returned error can't find the container with id c13767ed15f59e306219fd7caaf739ddb329e744a162ab23fde41b3101e85291
Nov 01 22:44:13.405985 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:13.405933    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ca20de96-6023-4e74-b8f1-d65d2123e913" path="/var/lib/kubelet/pods/ca20de96-6023-4e74-b8f1-d65d2123e913/volumes"
Nov 01 22:44:13.564572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:13.563831    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:13.970711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:13.970674    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" event={"ID":"5eb7ec26-3c0c-414c-995c-503eeb590770","Type":"ContainerStarted","Data":"c13767ed15f59e306219fd7caaf739ddb329e744a162ab23fde41b3101e85291"}
Nov 01 22:44:14.559077 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:14.559040    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:15.018280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.018199    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" event={"ID":"5eb7ec26-3c0c-414c-995c-503eeb590770","Type":"ContainerStarted","Data":"c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581"}
Nov 01 22:44:15.018652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.018622    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:15.065552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.065158    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" podStartSLOduration=3.635733583 podStartE2EDuration="4.065139373s" totalImagesPullingTime="429.40579ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:44:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:44:15.061115555 +0000 UTC m=+1311.802811175" watchObservedRunningTime="2026-11-01 22:44:15.065139373 +0000 UTC m=+1311.806834993"
Nov 01 22:44:15.358728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.358645    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:15.507061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.507027    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:15.628265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.628184    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"]
Nov 01 22:44:15.629005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.628970    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" containerID="cri-o://de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168" gracePeriod=30
Nov 01 22:44:15.645374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.644853    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"]
Nov 01 22:44:15.645374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.645060    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" containerID="cri-o://5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530" gracePeriod=30
Nov 01 22:44:15.660639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.660616    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"]
Nov 01 22:44:15.661003 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.660978    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy" containerID="cri-o://42e84539a8fb89d414a3c79ab9689fc9780667692458c60796b94f24e58c5dcc" gracePeriod=30
Nov 01 22:44:15.676551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.671977    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:15.679169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.679144    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"]
Nov 01 22:44:15.679393 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.679372    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy" containerID="cri-o://5d5edab34004c55f36f0ffc2dd1adcb22859bd6e2feb80147b052587e0b839ba" gracePeriod=30
Nov 01 22:44:15.841518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:15.841050    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:16.397863 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.397826    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:16.515541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.515467    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:44:16.515709 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.515572    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:44:16.560293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.560257    5057 patch_prober.go:29] interesting pod/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:44:16.560481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.560333    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:44:16.597315 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.597276    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:16.903050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:16.899208    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:17.665236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:17.663232    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:17.838597 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:17.838321    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:18.478546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:18.477831    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:18.618205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:18.618170    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:18.814342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:18.814252    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:19.788005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:19.787960    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:20.023672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.018744    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:20.328543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.327956    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:20.378722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.378686    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"]
Nov 01 22:44:20.380315 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.380066    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy" containerID="cri-o://a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3" gracePeriod=30
Nov 01 22:44:20.401906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.401877    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"]
Nov 01 22:44:20.402124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.402101    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy" containerID="cri-o://0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7" gracePeriod=30
Nov 01 22:44:20.511521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:20.502326    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:21.064578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.042840    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:44:21.107370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.093574    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:44:21.107370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.106668    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:21.205014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.204983    5057 generic.go:356] "Generic (PLEG): container finished" podID="e8586610-fbf5-4c12-a557-33747da52fc3" containerID="5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530" exitCode=0
Nov 01 22:44:21.205505 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.205273    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"
Nov 01 22:44:21.206395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.205308    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" event={"ID":"e8586610-fbf5-4c12-a557-33747da52fc3","Type":"ContainerDied","Data":"5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530"}
Nov 01 22:44:21.206875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.206854    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v" event={"ID":"e8586610-fbf5-4c12-a557-33747da52fc3","Type":"ContainerDied","Data":"d2773376a866bdd3615e17ae127cecc85b0787861b33b0f7faef9d37f59dd5bb"}
Nov 01 22:44:21.207179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.206947    5057 scope.go:122] "RemoveContainer" containerID="5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530"
Nov 01 22:44:21.207982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.207952    5057 generic.go:356] "Generic (PLEG): container finished" podID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerID="5d5edab34004c55f36f0ffc2dd1adcb22859bd6e2feb80147b052587e0b839ba" exitCode=0
Nov 01 22:44:21.208069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.208043    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" event={"ID":"7292edd4-bdf6-4f1e-be5d-f41670c8a84f","Type":"ContainerDied","Data":"5d5edab34004c55f36f0ffc2dd1adcb22859bd6e2feb80147b052587e0b839ba"}
Nov 01 22:44:21.211558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.208751    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:44:21.216249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.216225    5057 generic.go:356] "Generic (PLEG): container finished" podID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerID="de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168" exitCode=0
Nov 01 22:44:21.216377 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.216331    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" event={"ID":"45732ef3-3377-4231-bb8b-54f77cb7cf5c","Type":"ContainerDied","Data":"de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168"}
Nov 01 22:44:21.216459 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.216382    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq" event={"ID":"45732ef3-3377-4231-bb8b-54f77cb7cf5c","Type":"ContainerDied","Data":"5c33f5a2e7dd3e0beaeca5a9d0d4555ee3958548acd384618e491ab44606a7d7"}
Nov 01 22:44:21.218197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.218179    5057 generic.go:356] "Generic (PLEG): container finished" podID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerID="42e84539a8fb89d414a3c79ab9689fc9780667692458c60796b94f24e58c5dcc" exitCode=0
Nov 01 22:44:21.218305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.218243    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" event={"ID":"b2ab5df1-4570-4aed-a39f-c3faee5a8df9","Type":"ContainerDied","Data":"42e84539a8fb89d414a3c79ab9689fc9780667692458c60796b94f24e58c5dcc"}
Nov 01 22:44:21.272685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.272650    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.279965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.279936    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.280188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280172    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.280325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280310    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.280450 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280437    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.280592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280578    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.280733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280719    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.280842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280829    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.280939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.280927    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.281041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.281029    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.281272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.281257    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.281448 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.281434    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.283163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.283142    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.283430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.283413    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.283571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.283556    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.283690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.283677    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.283811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.283797    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data\") pod \"e8586610-fbf5-4c12-a557-33747da52fc3\" (UID: \"e8586610-fbf5-4c12-a557-33747da52fc3\") "
Nov 01 22:44:21.283942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.283930    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket\") pod \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\" (UID: \"45732ef3-3377-4231-bb8b-54f77cb7cf5c\") "
Nov 01 22:44:21.287461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.273205    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:21.287610 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.287154    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.289611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.289433    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.289611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.289486    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.290603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.276875    5057 scope.go:122] "RemoveContainer" containerID="5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530"
Nov 01 22:44:21.291094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.290773    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.291717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.291444    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.291717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.291623    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:21.291717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.291664    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.296715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.293451    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:44:21.296715 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:21.295145    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530\": container with ID starting with 5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530 not found: ID does not exist" containerID="5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530"
Nov 01 22:44:21.296715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.295184    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530"} err="failed to get container status \"5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530\": rpc error: code = NotFound desc = could not find container \"5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530\": container with ID starting with 5de704fe8e6aed0a88e1739e6a553c5e86178fd65e55e97c66c13624ff81a530 not found: ID does not exist"
Nov 01 22:44:21.296715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.295211    5057 scope.go:122] "RemoveContainer" containerID="de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168"
Nov 01 22:44:21.296715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.296076    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.296715 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.296199    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "kube-api-access-4kp4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.304189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.303637    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.312531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.311115    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.312531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.312331    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:44:21.330350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.330154    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.330547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.330516    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "kube-api-access-xmkn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.330631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.330558    5057 scope.go:122] "RemoveContainer" containerID="de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168"
Nov 01 22:44:21.330756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.330731    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy" pod "45732ef3-3377-4231-bb8b-54f77cb7cf5c" (UID: "45732ef3-3377-4231-bb8b-54f77cb7cf5c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.331147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.331049    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token" pod "e8586610-fbf5-4c12-a557-33747da52fc3" (UID: "e8586610-fbf5-4c12-a557-33747da52fc3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.331403 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:21.331377    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168\": container with ID starting with de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168 not found: ID does not exist" containerID="de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168"
Nov 01 22:44:21.331767 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.331563    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168"} err="failed to get container status \"de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168\": rpc error: code = NotFound desc = could not find container \"de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168\": container with ID starting with de4078ffc094d9af45acc20369a2a54acb6cc592b5a4ba917554bb7e5e6b6168 not found: ID does not exist"
Nov 01 22:44:21.339156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.339130    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:44:21.343341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.343265    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:44:21.386862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.386832    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.386870    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.386891    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.386929    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.386956    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387014    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387072    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387102    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387150    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387181    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387215    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387244    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387279    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387348    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387378    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387414    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.387735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387453    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data\") pod \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\" (UID: \"b2ab5df1-4570-4aed-a39f-c3faee5a8df9\") "
Nov 01 22:44:21.387735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.387531    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs\") pod \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\" (UID: \"7292edd4-bdf6-4f1e-be5d-f41670c8a84f\") "
Nov 01 22:44:21.388635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.388610    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.388800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.388787    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmkn6\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-kube-api-access-xmkn6\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.388890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.388879    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.388968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.388958    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389029    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389102    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389192 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389182    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389257    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389330    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389421 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389409    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389613    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389718    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.389816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389804    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.390041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389900    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8586610-fbf5-4c12-a557-33747da52fc3-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.390147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390127    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45732ef3-3377-4231-bb8b-54f77cb7cf5c-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.390217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390150    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8586610-fbf5-4c12-a557-33747da52fc3-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.390217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390165    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8586610-fbf5-4c12-a557-33747da52fc3-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.390217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390180    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kp4l\" (UniqueName: \"kubernetes.io/projected/e8586610-fbf5-4c12-a557-33747da52fc3-kube-api-access-4kp4l\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.390217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.388658    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.390217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389124    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:21.390217 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.389792    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.390606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390447    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.390756 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390739    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:21.390967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.390951    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.391285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.391230    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.391568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.391543    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.391724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.391622    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.391897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.391876    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.393765 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.393739    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.394387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.394334    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "kube-api-access-tjnbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.395857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.395832    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "kube-api-access-k99zs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.396246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.396225    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:44:21.400700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.399394    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:44:21.402128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.402100    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.402235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.402176    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy" pod "b2ab5df1-4570-4aed-a39f-c3faee5a8df9" (UID: "b2ab5df1-4570-4aed-a39f-c3faee5a8df9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:21.402300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.402278    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token" pod "7292edd4-bdf6-4f1e-be5d-f41670c8a84f" (UID: "7292edd4-bdf6-4f1e-be5d-f41670c8a84f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:21.491986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491923    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.491986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491950    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.491986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491959    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.491986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491968    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.491986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491982    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491991    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.491999    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492007    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492016    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492026    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492036    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjnbh\" (UniqueName: \"kubernetes.io/projected/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-kube-api-access-tjnbh\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492045    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492057    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492067    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7292edd4-bdf6-4f1e-be5d-f41670c8a84f-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492076    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492097    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492120    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.492221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.492130    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k99zs\" (UniqueName: \"kubernetes.io/projected/b2ab5df1-4570-4aed-a39f-c3faee5a8df9-kube-api-access-k99zs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:21.525629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.525603    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"]
Nov 01 22:44:21.531013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:21.530987    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-b654v"]
Nov 01 22:44:22.082403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.082371    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:22.232355 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.232323    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6" event={"ID":"7292edd4-bdf6-4f1e-be5d-f41670c8a84f","Type":"ContainerDied","Data":"5509692c0f7a838fb96a2c2a29b6d023bfc5d009dc86fa289dc57d78b04d298a"}
Nov 01 22:44:22.232580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.232397    5057 scope.go:122] "RemoveContainer" containerID="5d5edab34004c55f36f0ffc2dd1adcb22859bd6e2feb80147b052587e0b839ba"
Nov 01 22:44:22.232580 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.232331    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"
Nov 01 22:44:22.234073 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.233969    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"
Nov 01 22:44:22.236159 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.236133    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2" event={"ID":"b2ab5df1-4570-4aed-a39f-c3faee5a8df9","Type":"ContainerDied","Data":"2fcb7f450b65a04125cc50ae2328e3ebda72eb7237e6c1116ae92122b61689a8"}
Nov 01 22:44:22.236282 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.236223    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"
Nov 01 22:44:22.246894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.246864    5057 scope.go:122] "RemoveContainer" containerID="42e84539a8fb89d414a3c79ab9689fc9780667692458c60796b94f24e58c5dcc"
Nov 01 22:44:22.263370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.263343    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"]
Nov 01 22:44:22.270093 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.269922    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-sbpw2"]
Nov 01 22:44:22.278047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.278023    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"]
Nov 01 22:44:22.282261 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.282237    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-p56wq"]
Nov 01 22:44:22.291934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.291911    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"]
Nov 01 22:44:22.297214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.297193    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a1d4be-openshift-default-85f677f455-6n2q6"]
Nov 01 22:44:22.756694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.756655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:22.954546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:22.952151    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:23.401886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.401850    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" path="/var/lib/kubelet/pods/45732ef3-3377-4231-bb8b-54f77cb7cf5c/volumes"
Nov 01 22:44:23.402417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.402402    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" path="/var/lib/kubelet/pods/7292edd4-bdf6-4f1e-be5d-f41670c8a84f/volumes"
Nov 01 22:44:23.402956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.402941    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" path="/var/lib/kubelet/pods/b2ab5df1-4570-4aed-a39f-c3faee5a8df9/volumes"
Nov 01 22:44:23.403750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.403735    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" path="/var/lib/kubelet/pods/e8586610-fbf5-4c12-a557-33747da52fc3/volumes"
Nov 01 22:44:23.540513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.539151    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:23.746453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.746372    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:23.881328 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:23.881298    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:24.301544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:24.300232    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:24.971157 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:24.971119    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:25.809520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:25.807312    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:26.067131 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.067096    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:44:26.103586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.103542    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:44:26.238370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238339    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238385    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238416    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238442    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238465    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238507    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238570    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238597    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238626    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238670    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.238793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238781    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238846    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238902    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238951    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.238999    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.239037    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.239066    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.239082    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket\") pod \"daead9d7-2777-4ed1-943d-382abf76fe36\" (UID: \"daead9d7-2777-4ed1-943d-382abf76fe36\") "
Nov 01 22:44:26.239604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.239126    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo\") pod \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\" (UID: \"7792407c-331d-46d3-b2a3-7b7d2c48d5fa\") "
Nov 01 22:44:26.240065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.239870    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.240702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.240673    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.240826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.240783    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.240913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.240891    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.240988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.240916    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.240988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.240974    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:26.241370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.241327    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:44:26.242249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.242164    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.242249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.242224    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.242707 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.242578    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.243087 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.243052    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "kube-api-access-sbgv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:26.244072 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.244046    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.244161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.244091    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:44:26.244161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.244101    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:44:26.244161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.244118    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:26.244267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.244185    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy" pod "7792407c-331d-46d3-b2a3-7b7d2c48d5fa" (UID: "7792407c-331d-46d3-b2a3-7b7d2c48d5fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.245388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.245364    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "kube-api-access-l95hk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:26.245810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.245787    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy" pod "daead9d7-2777-4ed1-943d-382abf76fe36" (UID: "daead9d7-2777-4ed1-943d-382abf76fe36"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341603    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341640    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l95hk\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-kube-api-access-l95hk\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341656    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341671    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341685    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341699    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341715    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341730    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341744    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/daead9d7-2777-4ed1-943d-382abf76fe36-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341758    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/daead9d7-2777-4ed1-943d-382abf76fe36-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341773    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341786    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/daead9d7-2777-4ed1-943d-382abf76fe36-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341801    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/daead9d7-2777-4ed1-943d-382abf76fe36-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341815    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341831    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341845    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.346565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.341859    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbgv6\" (UniqueName: \"kubernetes.io/projected/7792407c-331d-46d3-b2a3-7b7d2c48d5fa-kube-api-access-sbgv6\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:26.400021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.397904    5057 generic.go:356] "Generic (PLEG): container finished" podID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerID="0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7" exitCode=0
Nov 01 22:44:26.400021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.398149    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" event={"ID":"7792407c-331d-46d3-b2a3-7b7d2c48d5fa","Type":"ContainerDied","Data":"0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7"}
Nov 01 22:44:26.400021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.398207    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x" event={"ID":"7792407c-331d-46d3-b2a3-7b7d2c48d5fa","Type":"ContainerDied","Data":"2a7c77ec4de98d166a8d5b567607b8b96e845a744d3f9ed904668f5747e221c0"}
Nov 01 22:44:26.400021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.398237    5057 scope.go:122] "RemoveContainer" containerID="0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7"
Nov 01 22:44:26.400021 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.398394    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"
Nov 01 22:44:26.403194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.403165    5057 generic.go:356] "Generic (PLEG): container finished" podID="daead9d7-2777-4ed1-943d-382abf76fe36" containerID="a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3" exitCode=0
Nov 01 22:44:26.403313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.403224    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"
Nov 01 22:44:26.403313 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.403258    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" event={"ID":"daead9d7-2777-4ed1-943d-382abf76fe36","Type":"ContainerDied","Data":"a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3"}
Nov 01 22:44:26.403416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.403326    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff" event={"ID":"daead9d7-2777-4ed1-943d-382abf76fe36","Type":"ContainerDied","Data":"88180b71ffeae2b96a8a478ecdb5ede3b73de1ca89d4b8aa1f26606f78995b5a"}
Nov 01 22:44:26.414649 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.414619    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:26.438953 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.438926    5057 scope.go:122] "RemoveContainer" containerID="0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7"
Nov 01 22:44:26.439357 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:26.439333    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7\": container with ID starting with 0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7 not found: ID does not exist" containerID="0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7"
Nov 01 22:44:26.439468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.439368    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7"} err="failed to get container status \"0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7\": rpc error: code = NotFound desc = could not find container \"0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7\": container with ID starting with 0c2319b4f3ad1e4bf1829d09ab1fac30e2c5488d4b411eedd3b6591644c560b7 not found: ID does not exist"
Nov 01 22:44:26.439468 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.439395    5057 scope.go:122] "RemoveContainer" containerID="a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3"
Nov 01 22:44:26.453382 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.453352    5057 scope.go:122] "RemoveContainer" containerID="a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3"
Nov 01 22:44:26.453818 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:26.453785    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3\": container with ID starting with a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3 not found: ID does not exist" containerID="a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3"
Nov 01 22:44:26.453918 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.453829    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3"} err="failed to get container status \"a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3\": rpc error: code = NotFound desc = could not find container \"a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3\": container with ID starting with a60f4681227fc841593e575bb1cd8558a2adc88297247908dfdddc5f4b546fd3 not found: ID does not exist"
Nov 01 22:44:26.462560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.457941    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"]
Nov 01 22:44:26.463643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.463618    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-pm59x"]
Nov 01 22:44:26.471618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.471590    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"]
Nov 01 22:44:26.484413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.484378    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-ee8d1a-openshift-default-6bc8466c9c-tzqff"]
Nov 01 22:44:26.564196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.559418    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:26.940196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:26.940157    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:27.472160 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:27.472122    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" path="/var/lib/kubelet/pods/7792407c-331d-46d3-b2a3-7b7d2c48d5fa/volumes"
Nov 01 22:44:27.475681 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:27.475649    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" path="/var/lib/kubelet/pods/daead9d7-2777-4ed1-943d-382abf76fe36/volumes"
Nov 01 22:44:27.607515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:27.607171    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" containerID="cri-o://382fe443abef6f43684af63e100466060796f0fb79583c3fb08e467e3ef2f292" gracePeriod=45
Nov 01 22:44:27.692526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:27.687958    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": dial tcp 10.132.0.52:8000: connect: connection refused"
Nov 01 22:44:28.731851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:28.650060    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:28.731851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:28.661646    5057 generic.go:356] "Generic (PLEG): container finished" podID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerID="382fe443abef6f43684af63e100466060796f0fb79583c3fb08e467e3ef2f292" exitCode=0
Nov 01 22:44:28.731851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:28.661928    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" event={"ID":"01d597ea-da12-43c9-bd1a-dd26d47fd332","Type":"ContainerDied","Data":"382fe443abef6f43684af63e100466060796f0fb79583c3fb08e467e3ef2f292"}
Nov 01 22:44:28.850515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:28.840447    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"]
Nov 01 22:44:29.039539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.030604    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:29.087020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.086576    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:29.121551 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.120979    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:29.284602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.284526    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:44:29.418305 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.418264    5057 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/health\": context deadline exceeded"
Nov 01 22:44:29.472520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.472451    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw\" (UniqueName: \"kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw\") pod \"01d597ea-da12-43c9-bd1a-dd26d47fd332\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") "
Nov 01 22:44:29.472774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.472755    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm\") pod \"01d597ea-da12-43c9-bd1a-dd26d47fd332\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") "
Nov 01 22:44:29.473111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.473093    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir\") pod \"01d597ea-da12-43c9-bd1a-dd26d47fd332\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") "
Nov 01 22:44:29.473271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.473248    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache\") pod \"01d597ea-da12-43c9-bd1a-dd26d47fd332\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") "
Nov 01 22:44:29.473487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.473474    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home\") pod \"01d597ea-da12-43c9-bd1a-dd26d47fd332\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") "
Nov 01 22:44:29.473788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.473768    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs\" (UniqueName: \"kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs\") pod \"01d597ea-da12-43c9-bd1a-dd26d47fd332\" (UID: \"01d597ea-da12-43c9-bd1a-dd26d47fd332\") "
Nov 01 22:44:29.476028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.476005    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home" pod "01d597ea-da12-43c9-bd1a-dd26d47fd332" (UID: "01d597ea-da12-43c9-bd1a-dd26d47fd332"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:29.476224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.476205    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache" pod "01d597ea-da12-43c9-bd1a-dd26d47fd332" (UID: "01d597ea-da12-43c9-bd1a-dd26d47fd332"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:29.476365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.476349    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir" pod "01d597ea-da12-43c9-bd1a-dd26d47fd332" (UID: "01d597ea-da12-43c9-bd1a-dd26d47fd332"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:29.587294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.585662    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs" pod "01d597ea-da12-43c9-bd1a-dd26d47fd332" (UID: "01d597ea-da12-43c9-bd1a-dd26d47fd332"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:44:29.587294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.587185    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:29.587294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.587216    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:29.587294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.587231    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:29.587294 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.587259    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/01d597ea-da12-43c9-bd1a-dd26d47fd332-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:29.629980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.629925    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm" pod "01d597ea-da12-43c9-bd1a-dd26d47fd332" (UID: "01d597ea-da12-43c9-bd1a-dd26d47fd332"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:29.697847 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.697127    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" probeResult="failure" output="Get \"https://10.132.0.61:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:29.713520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.702462    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main" probeResult="failure" output="Get \"https://10.132.0.52:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:29.713520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.702991    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:44:29.717746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.717715    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/01d597ea-da12-43c9-bd1a-dd26d47fd332-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:29.731740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.731703    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw" pod "01d597ea-da12-43c9-bd1a-dd26d47fd332" (UID: "01d597ea-da12-43c9-bd1a-dd26d47fd332"). InnerVolumeSpecName "kube-api-access-z26qw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:29.775524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.751454    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7" event={"ID":"01d597ea-da12-43c9-bd1a-dd26d47fd332","Type":"ContainerDied","Data":"5226dd15018f77f7d6e3432e2eb027f9168a8dc72e96c5119bb19ab70e6383f0"}
Nov 01 22:44:29.775524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.751533    5057 scope.go:122] "RemoveContainer" containerID="382fe443abef6f43684af63e100466060796f0fb79583c3fb08e467e3ef2f292"
Nov 01 22:44:29.775524 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.751697    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"
Nov 01 22:44:29.890520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.867819    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z26qw\" (UniqueName: \"kubernetes.io/projected/01d597ea-da12-43c9-bd1a-dd26d47fd332-kube-api-access-z26qw\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:29.908417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.908346    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:29.939161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:29.939131    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"]
Nov 01 22:44:30.040521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:30.038784    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-e25182/simulator-w0-e25182-kserve-7bb544d64b-wctg7"]
Nov 01 22:44:30.876516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:30.876295    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.70:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:31.020514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.015474    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:31.025525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.023197    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:31.046521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.042583    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": context deadline exceeded"
Nov 01 22:44:31.133574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.126593    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" probeResult="failure" output="Get \"https://10.132.0.71:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:31.210277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.204143    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" podUID="25c2a203-49a6-4361-bec4-8fb7f3357c33" containerName="main" probeResult="failure" output="Get \"https://10.132.0.92:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:44:31.220514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.215570    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded"
Nov 01 22:44:31.220514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.215898    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": context deadline exceeded"
Nov 01 22:44:31.381290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.380323    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:31.454599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.449142    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" path="/var/lib/kubelet/pods/01d597ea-da12-43c9-bd1a-dd26d47fd332/volumes"
Nov 01 22:44:31.541043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:31.541011    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:32.211029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:32.210995    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:32.989978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:32.989947    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:33.950757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:33.950502    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:34.662467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:34.662425    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:35.561488 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:35.561457    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:36.551296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:36.551162    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:36.719535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:36.719410    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:36.850282 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:36.850146    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:37.481667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:37.481543    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:38.363462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:38.363425    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:38.496829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:38.496791    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:39.285352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.281812    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:39.913042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.913013    5057 generic.go:356] "Generic (PLEG): container finished" podID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerID="6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199" exitCode=0
Nov 01 22:44:39.913442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.913081    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" event={"ID":"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7","Type":"ContainerDied","Data":"6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"}
Nov 01 22:44:39.913528 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.913457    5057 scope.go:122] "RemoveContainer" containerID="6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"
Nov 01 22:44:39.915165 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.915134    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:44:39.915236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.915200    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:44:39.915236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:39.915232    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:44:40.034845 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:40.034535    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:44:40.277398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:40.277360    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:40.704586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:40.704558    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:44:40.924110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:40.924065    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" event={"ID":"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7","Type":"ContainerStarted","Data":"114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b"}
Nov 01 22:44:41.090109 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:41.090014    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:41.340991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:41.340895    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:44:41.345639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:41.345605    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:44:41.345879 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:41.345847    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:44:41.897451 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:41.897410    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:42.348953 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:42.348916    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:44:42.349590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:42.349549    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:44:42.608193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:42.608086    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:43.341945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:43.341910    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:44:43.342210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:43.342182    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:44:43.594761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:43.594680    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:44.259098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.259042    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" podUID="25c2a203-49a6-4361-bec4-8fb7f3357c33" containerName="main" containerID="cri-o://a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f" gracePeriod=45
Nov 01 22:44:44.344409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.343964    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:44:44.344409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.344030    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:44:44.501968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.501578    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:44.713555 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.713526    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:44:44.809105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809076    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs\" (UniqueName: \"kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs\") pod \"25c2a203-49a6-4361-bec4-8fb7f3357c33\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") "
Nov 01 22:44:44.809268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809130    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir\") pod \"25c2a203-49a6-4361-bec4-8fb7f3357c33\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") "
Nov 01 22:44:44.809268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809213    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt\" (UniqueName: \"kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt\") pod \"25c2a203-49a6-4361-bec4-8fb7f3357c33\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") "
Nov 01 22:44:44.809268 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809260    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home\") pod \"25c2a203-49a6-4361-bec4-8fb7f3357c33\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") "
Nov 01 22:44:44.809449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809374    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm\") pod \"25c2a203-49a6-4361-bec4-8fb7f3357c33\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") "
Nov 01 22:44:44.809525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809453    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache\") pod \"25c2a203-49a6-4361-bec4-8fb7f3357c33\" (UID: \"25c2a203-49a6-4361-bec4-8fb7f3357c33\") "
Nov 01 22:44:44.810529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.809999    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache" pod "25c2a203-49a6-4361-bec4-8fb7f3357c33" (UID: "25c2a203-49a6-4361-bec4-8fb7f3357c33"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:44.810529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.810143    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir" pod "25c2a203-49a6-4361-bec4-8fb7f3357c33" (UID: "25c2a203-49a6-4361-bec4-8fb7f3357c33"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:44.810529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.810416    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:44.810529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.810438    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:44.810900 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.810818    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home" pod "25c2a203-49a6-4361-bec4-8fb7f3357c33" (UID: "25c2a203-49a6-4361-bec4-8fb7f3357c33"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:44.812721 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.812651    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm" pod "25c2a203-49a6-4361-bec4-8fb7f3357c33" (UID: "25c2a203-49a6-4361-bec4-8fb7f3357c33"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:44:44.812816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.812778    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs" pod "25c2a203-49a6-4361-bec4-8fb7f3357c33" (UID: "25c2a203-49a6-4361-bec4-8fb7f3357c33"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:44:44.812976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.812959    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt" pod "25c2a203-49a6-4361-bec4-8fb7f3357c33" (UID: "25c2a203-49a6-4361-bec4-8fb7f3357c33"). InnerVolumeSpecName "kube-api-access-4bptt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:44:44.911674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.911639    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25c2a203-49a6-4361-bec4-8fb7f3357c33-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:44.911674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.911667    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4bptt\" (UniqueName: \"kubernetes.io/projected/25c2a203-49a6-4361-bec4-8fb7f3357c33-kube-api-access-4bptt\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:44.911674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.911677    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:44.911914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.911685    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25c2a203-49a6-4361-bec4-8fb7f3357c33-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:44:44.995007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.994967    5057 generic.go:356] "Generic (PLEG): container finished" podID="25c2a203-49a6-4361-bec4-8fb7f3357c33" containerID="a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f" exitCode=0
Nov 01 22:44:44.995216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.995014    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"
Nov 01 22:44:44.995216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.995046    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" event={"ID":"25c2a203-49a6-4361-bec4-8fb7f3357c33","Type":"ContainerDied","Data":"a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f"}
Nov 01 22:44:44.995216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.995091    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5" event={"ID":"25c2a203-49a6-4361-bec4-8fb7f3357c33","Type":"ContainerDied","Data":"bc4e483f3faab9c75d2cbaac1826eec3da8017eb30d9908e4b144e8f14105744"}
Nov 01 22:44:44.995216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:44.995109    5057 scope.go:122] "RemoveContainer" containerID="a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f"
Nov 01 22:44:45.014237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.014216    5057 scope.go:122] "RemoveContainer" containerID="a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f"
Nov 01 22:44:45.014610 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:44:45.014587    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f\": container with ID starting with a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f not found: ID does not exist" containerID="a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f"
Nov 01 22:44:45.014683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.014621    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f"} err="failed to get container status \"a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f\": rpc error: code = NotFound desc = could not find container \"a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f\": container with ID starting with a352bbb6d3839582998a129c83beaf40a25eed018a0e43d7cc17140bc747ce7f not found: ID does not exist"
Nov 01 22:44:45.056577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.056536    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"]
Nov 01 22:44:45.060449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.060416    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-425870c1/e2e-lifecycle-model-425870c1-kserve-84d75db6b9-crhq5"]
Nov 01 22:44:45.213550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.213475    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:45.354224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.354134    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:44:45.354559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.354469    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:44:45.399981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:45.399947    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25c2a203-49a6-4361-bec4-8fb7f3357c33" path="/var/lib/kubelet/pods/25c2a203-49a6-4361-bec4-8fb7f3357c33/volumes"
Nov 01 22:44:46.057651 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:46.057617    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:46.340930 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:46.340834    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused" start-of-body=
Nov 01 22:44:46.341185 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:46.341151    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.105:15021/healthz/ready\": dial tcp 10.132.0.105:15021: connect: connection refused"
Nov 01 22:44:46.786834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:46.786799    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:47.346053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:47.346022    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:44:47.346545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:47.346449    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:44:47.347317 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:47.347298    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:44:47.523343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:47.523310    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:48.071924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:48.071817    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:48.296514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:48.294883    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:48.639866 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:48.639835    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:48.869907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:48.869866    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:49.066133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:49.066101    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:49.716336 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:49.716299    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:50.061320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:50.061219    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:50.892095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:50.892037    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:51.411777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:51.411738    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:51.553286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:51.553255    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:51.966547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:51.966506    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:52.999215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:52.999172    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:53.200548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:53.200460    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:53.795212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:53.795178    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:55.088149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:55.088113    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:56.429130 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:56.429084    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:56.962708 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:56.962661    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:57.234654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:57.234561    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:57.449197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:57.449164    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:57.603169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:57.603076    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:57.766940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:57.766631    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:57.956286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:57.956244    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:58.249945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:58.249855    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:58.458372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:58.458333    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:58.679409 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:58.679310    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:59.343485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:59.343449    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:44:59.633105 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:44:59.633018    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:00.380302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.380262    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"]
Nov 01 22:45:00.381880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.381836    5057 cpu_manager.go:414] "containerMap: removing container" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy"
Nov 01 22:45:00.382058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382041    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy"
Nov 01 22:45:00.382204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382191    5057 cpu_manager.go:414] "containerMap: removing container" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy"
Nov 01 22:45:00.382316 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382303    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy"
Nov 01 22:45:00.382458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382446    5057 cpu_manager.go:414] "containerMap: removing container" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy"
Nov 01 22:45:00.382585 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382572    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy"
Nov 01 22:45:00.382692 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382681    5057 cpu_manager.go:414] "containerMap: removing container" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main"
Nov 01 22:45:00.382794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382782    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main"
Nov 01 22:45:00.382924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.382901    5057 cpu_manager.go:414] "containerMap: removing container" podUID="25c2a203-49a6-4361-bec4-8fb7f3357c33" containerName="main"
Nov 01 22:45:00.383031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383020    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25c2a203-49a6-4361-bec4-8fb7f3357c33" containerName="main"
Nov 01 22:45:00.383169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383157    5057 cpu_manager.go:414] "containerMap: removing container" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy"
Nov 01 22:45:00.383277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383265    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy"
Nov 01 22:45:00.383407 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383395    5057 cpu_manager.go:414] "containerMap: removing container" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy"
Nov 01 22:45:00.383519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383507    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy"
Nov 01 22:45:00.383640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383629    5057 cpu_manager.go:414] "containerMap: removing container" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy"
Nov 01 22:45:00.383740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.383728    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy"
Nov 01 22:45:00.384068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384041    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="25c2a203-49a6-4361-bec4-8fb7f3357c33" containerName="main"
Nov 01 22:45:00.384199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384188    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="45732ef3-3377-4231-bb8b-54f77cb7cf5c" containerName="istio-proxy"
Nov 01 22:45:00.384344 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384331    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="daead9d7-2777-4ed1-943d-382abf76fe36" containerName="istio-proxy"
Nov 01 22:45:00.384449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384439    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="7292edd4-bdf6-4f1e-be5d-f41670c8a84f" containerName="istio-proxy"
Nov 01 22:45:00.384576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384564    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8586610-fbf5-4c12-a557-33747da52fc3" containerName="istio-proxy"
Nov 01 22:45:00.384693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384682    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2ab5df1-4570-4aed-a39f-c3faee5a8df9" containerName="istio-proxy"
Nov 01 22:45:00.384801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384790    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="7792407c-331d-46d3-b2a3-7b7d2c48d5fa" containerName="istio-proxy"
Nov 01 22:45:00.384975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.384963    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="01d597ea-da12-43c9-bd1a-dd26d47fd332" containerName="main"
Nov 01 22:45:00.391133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.391111    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.406436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.406411    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xlj2w\""
Nov 01 22:45:00.407009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.406985    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"]
Nov 01 22:45:00.424196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.413230    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"]
Nov 01 22:45:00.470937 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.470900    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.472057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.470794    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"]
Nov 01 22:45:00.553427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.553388    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.553427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.553430    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qv8c2\" (UniqueName: \"kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2\") pod \"maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.554141 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.554108    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ll222\" (UniqueName: \"kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.554383 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.554359    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.655700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.655618    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ll222\" (UniqueName: \"kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.656179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.655699    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.656179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.655783    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.656179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.655812    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qv8c2\" (UniqueName: \"kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2\") pod \"maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.656666 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.656637    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.656796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.656641    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.666565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.666538    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qv8c2\" (UniqueName: \"kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2\") pod \"maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.666691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.666546    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ll222\" (UniqueName: \"kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222\") pod \"maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.748696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.748660    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:00.793816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.793779    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:00.820515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:00.816743    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:01.091467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:01.089756    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"]
Nov 01 22:45:01.094128 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:45:01.094099    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0305b796_0ef6_47ef_9c93_2359677f72f8.slice/crio-d35ba31e9e64f247bb8b8d1780dafdc4c0833991edffbc578e98ee3c3beed4b3 WatchSource:0}: Error finding container d35ba31e9e64f247bb8b8d1780dafdc4c0833991edffbc578e98ee3c3beed4b3: Status 404 returned error can't find the container with id d35ba31e9e64f247bb8b8d1780dafdc4c0833991edffbc578e98ee3c3beed4b3
Nov 01 22:45:01.227684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:01.227650    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"]
Nov 01 22:45:01.278646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:01.278563    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerStarted","Data":"e7f0800126524d41e66866cb50f51d59a0195df95b8d9cad409692839ab19f47"}
Nov 01 22:45:01.280164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:01.280129    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerStarted","Data":"d35ba31e9e64f247bb8b8d1780dafdc4c0833991edffbc578e98ee3c3beed4b3"}
Nov 01 22:45:01.754460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:01.753189    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:02.248921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:02.248882    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:02.394780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:02.394730    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:02.866364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:02.866326    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:03.280948 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:03.280907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:03.449526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:03.444186    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:03.949454 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:03.949412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:04.218537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:04.210589    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:04.222393 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:04.222356    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"]
Nov 01 22:45:04.500554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:04.500457    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:04.754522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:04.754390    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:04.976361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:04.976328    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:05.764299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:05.764268    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:06.354587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:06.354540    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerStarted","Data":"1a9cc327b1bf2db3727e8977f502d477bc2ebefaafdb22b8281a6d3babe978bf"}
Nov 01 22:45:06.356571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:06.356532    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerStarted","Data":"e8e53f4580f3b7e0b450dc9f55e1a8f0138c928fdcf86e6015bd1c9ee90f189f"}
Nov 01 22:45:06.376858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:06.376809    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" podStartSLOduration=1.857576254 podStartE2EDuration="6.376797656s" totalImagesPullingTime="4.519221402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:45:06.372914593 +0000 UTC m=+1363.114610211" watchObservedRunningTime="2026-11-01 22:45:06.376797656 +0000 UTC m=+1363.118493308"
Nov 01 22:45:06.399856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:06.399769    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" podStartSLOduration=1.5485301360000001 podStartE2EDuration="6.399756855s" totalImagesPullingTime="4.851226719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:45:06.3953987 +0000 UTC m=+1363.137094319" watchObservedRunningTime="2026-11-01 22:45:06.399756855 +0000 UTC m=+1363.141452473"
Nov 01 22:45:06.791516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:06.787865    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:07.085090 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.084827    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:07.366712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.366615    5057 generic.go:356] "Generic (PLEG): container finished" podID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerID="e8e53f4580f3b7e0b450dc9f55e1a8f0138c928fdcf86e6015bd1c9ee90f189f" exitCode=60
Nov 01 22:45:07.367206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.366703    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerDied","Data":"e8e53f4580f3b7e0b450dc9f55e1a8f0138c928fdcf86e6015bd1c9ee90f189f"}
Nov 01 22:45:07.367206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.367050    5057 scope.go:122] "RemoveContainer" containerID="e8e53f4580f3b7e0b450dc9f55e1a8f0138c928fdcf86e6015bd1c9ee90f189f"
Nov 01 22:45:07.368702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.368678    5057 generic.go:356] "Generic (PLEG): container finished" podID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerID="1a9cc327b1bf2db3727e8977f502d477bc2ebefaafdb22b8281a6d3babe978bf" exitCode=60
Nov 01 22:45:07.368842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.368761    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerDied","Data":"1a9cc327b1bf2db3727e8977f502d477bc2ebefaafdb22b8281a6d3babe978bf"}
Nov 01 22:45:07.369067 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.369050    5057 scope.go:122] "RemoveContainer" containerID="1a9cc327b1bf2db3727e8977f502d477bc2ebefaafdb22b8281a6d3babe978bf"
Nov 01 22:45:07.490931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.490890    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"]
Nov 01 22:45:07.491171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.491144    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw" podUID="cd8aa617-b822-4f46-8bb2-6bdb371b9982" containerName="maas-api" containerID="cri-o://c713593ad372124973b406f86e4c8cba1664fa5287317a2db7f5146bfde830d5" gracePeriod=30
Nov 01 22:45:07.570547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.569894    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:07.817286 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:07.817117    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:08.214784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.214741    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:08.420566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.411463    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:08.506707 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.506672    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"]
Nov 01 22:45:08.507163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.507133    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" containerID="cri-o://1c6b7db0b161849223c1daa9e2e541840c3f55bd86bb97ce6dd5481fb58b1d8e" gracePeriod=30
Nov 01 22:45:08.525434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.524438    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"]
Nov 01 22:45:08.525434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.524536    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"]
Nov 01 22:45:08.525434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.524723    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" containerID="cri-o://4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0" gracePeriod=30
Nov 01 22:45:08.525434 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.524946    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy" containerID="cri-o://bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8" gracePeriod=30
Nov 01 22:45:08.557982 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.557948    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerStarted","Data":"ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393"}
Nov 01 22:45:08.559051 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.559029    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:45:08.560274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.560249    5057 generic.go:356] "Generic (PLEG): container finished" podID="cd8aa617-b822-4f46-8bb2-6bdb371b9982" containerID="c713593ad372124973b406f86e4c8cba1664fa5287317a2db7f5146bfde830d5" exitCode=0
Nov 01 22:45:08.560384 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.560344    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw" event={"ID":"cd8aa617-b822-4f46-8bb2-6bdb371b9982","Type":"ContainerDied","Data":"c713593ad372124973b406f86e4c8cba1664fa5287317a2db7f5146bfde830d5"}
Nov 01 22:45:08.560449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.560392    5057 scope.go:122] "RemoveContainer" containerID="c713593ad372124973b406f86e4c8cba1664fa5287317a2db7f5146bfde830d5"
Nov 01 22:45:08.565185 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.565149    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerStarted","Data":"f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2"}
Nov 01 22:45:08.581780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.581091    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:08.617523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.615945    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5\" (UniqueName: \"kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5\") pod \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") "
Nov 01 22:45:08.617523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.615996    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs\") pod \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") "
Nov 01 22:45:08.617523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.616113    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca\") pod \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") "
Nov 01 22:45:08.617523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.616271    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls\") pod \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\" (UID: \"cd8aa617-b822-4f46-8bb2-6bdb371b9982\") "
Nov 01 22:45:08.643890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.643846    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca" pod "cd8aa617-b822-4f46-8bb2-6bdb371b9982" (UID: "cd8aa617-b822-4f46-8bb2-6bdb371b9982"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:08.695790 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.695734    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs" pod "cd8aa617-b822-4f46-8bb2-6bdb371b9982" (UID: "cd8aa617-b822-4f46-8bb2-6bdb371b9982"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:08.740462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.738319    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:08.740462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.738354    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd8aa617-b822-4f46-8bb2-6bdb371b9982-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:08.800519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.800442    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5" pod "cd8aa617-b822-4f46-8bb2-6bdb371b9982" (UID: "cd8aa617-b822-4f46-8bb2-6bdb371b9982"). InnerVolumeSpecName "kube-api-access-rjjk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:08.840309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.839686    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls" pod "cd8aa617-b822-4f46-8bb2-6bdb371b9982" (UID: "cd8aa617-b822-4f46-8bb2-6bdb371b9982"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:08.844981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.844949    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd8aa617-b822-4f46-8bb2-6bdb371b9982-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:08.844981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.844984    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjjk5\" (UniqueName: \"kubernetes.io/projected/cd8aa617-b822-4f46-8bb2-6bdb371b9982-kube-api-access-rjjk5\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:08.863573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:08.861242    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:09.009357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.009316    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:09.029890 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:09.029852    5057 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86664679_f7d5_471e_b41b_98cdb6f48fdd.slice/crio-f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2.scope\": RecentStats: unable to find data in memory cache]"
Nov 01 22:45:09.573395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.573362    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"
Nov 01 22:45:09.573395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.573373    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw" event={"ID":"cd8aa617-b822-4f46-8bb2-6bdb371b9982","Type":"ContainerDied","Data":"ac50fe6494a7f3030b937a921627e7d9401c433dc1fe6798810baf89bbb7412e"}
Nov 01 22:45:09.575669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.575637    5057 generic.go:356] "Generic (PLEG): container finished" podID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerID="f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2" exitCode=60
Nov 01 22:45:09.575827 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.575803    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerDied","Data":"f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2"}
Nov 01 22:45:09.575899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.575856    5057 scope.go:122] "RemoveContainer" containerID="1a9cc327b1bf2db3727e8977f502d477bc2ebefaafdb22b8281a6d3babe978bf"
Nov 01 22:45:09.576245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.576225    5057 scope.go:122] "RemoveContainer" containerID="f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2"
Nov 01 22:45:09.576591 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:09.576571    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd_odh-ai-gateway-infra(86664679-f7d5-471e-b41b-98cdb6f48fdd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd"
Nov 01 22:45:09.578992 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.578967    5057 generic.go:356] "Generic (PLEG): container finished" podID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerID="ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393" exitCode=60
Nov 01 22:45:09.579100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.579064    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerDied","Data":"ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393"}
Nov 01 22:45:09.579380 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.579365    5057 scope.go:122] "RemoveContainer" containerID="ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393"
Nov 01 22:45:09.579716 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:09.579698    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds_odh-ai-gateway-infra(0305b796-0ef6-47ef-9c93-2359677f72f8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8"
Nov 01 22:45:09.590170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.590153    5057 scope.go:122] "RemoveContainer" containerID="e8e53f4580f3b7e0b450dc9f55e1a8f0138c928fdcf86e6015bd1c9ee90f189f"
Nov 01 22:45:09.620320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.620291    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"]
Nov 01 22:45:09.626417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.626388    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b4f6bb-78cfd78d76-24hnw"]
Nov 01 22:45:09.661341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.661285    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:09.868247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:09.868214    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:10.018009 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:10.017975    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:45:10.118546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.118350    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:10.255676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.255632    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:10.513125 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.513090    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:45:10.513339 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.513145    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:45:10.590226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.590191    5057 scope.go:122] "RemoveContainer" containerID="f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2"
Nov 01 22:45:10.591235 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:10.590618    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd_odh-ai-gateway-infra(86664679-f7d5-471e-b41b-98cdb6f48fdd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd"
Nov 01 22:45:10.593891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.593857    5057 scope.go:122] "RemoveContainer" containerID="ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393"
Nov 01 22:45:10.594284 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:10.594256    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds_odh-ai-gateway-infra(0305b796-0ef6-47ef-9c93-2359677f72f8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8"
Nov 01 22:45:10.775748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:10.775665    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:11.399153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:11.399120    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cd8aa617-b822-4f46-8bb2-6bdb371b9982" path="/var/lib/kubelet/pods/cd8aa617-b822-4f46-8bb2-6bdb371b9982/volumes"
Nov 01 22:45:11.783620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:11.783588    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:45:11.822923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:11.822878    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:13.044512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.040434    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:13.445465 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.445375    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:13.634865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.634825    5057 generic.go:356] "Generic (PLEG): container finished" podID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerID="1c6b7db0b161849223c1daa9e2e541840c3f55bd86bb97ce6dd5481fb58b1d8e" exitCode=0
Nov 01 22:45:13.635012 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.634905    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" event={"ID":"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1","Type":"ContainerDied","Data":"1c6b7db0b161849223c1daa9e2e541840c3f55bd86bb97ce6dd5481fb58b1d8e"}
Nov 01 22:45:13.794594 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.794513    5057 patch_prober.go:29] interesting pod/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Nov 01 22:45:13.794764 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.794595    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Nov 01 22:45:13.897543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.897514    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:45:13.908170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.908146    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:45:13.946455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946422    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.946628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946463    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.946628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946581    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.946628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946612    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.946828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946651    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.946828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946699    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.946828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946755    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.946828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946798    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946833    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946895    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946940    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946950    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.946997    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947004    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947056    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.947097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947098    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947151    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947161    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947182    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947198    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947263    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947304    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947361    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy\") pod \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\" (UID: \"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947413    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo\") pod \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\" (UID: \"d4257a9b-63b8-4ca3-896a-080c7fb3853f\") "
Nov 01 22:45:13.947560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947469    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.947842    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948252    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948276    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948290    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948307    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948322    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948335    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948348    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948362    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:13.949362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.948572    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.949925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.949578    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:45:13.949925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.949792    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "kube-api-access-9xnwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:13.949925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.949820    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.949925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.949825    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "kube-api-access-wrvs9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:13.950204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.950182    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:45:13.950839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.950819    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:13.950946 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.950836    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:13.951144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.951103    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token" pod "d4257a9b-63b8-4ca3-896a-080c7fb3853f" (UID: "d4257a9b-63b8-4ca3-896a-080c7fb3853f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:13.951651 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:13.951620    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy" pod "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" (UID: "f64b1efe-a50d-49d7-a4bb-0a98e2d052f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050148    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrvs9\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-kube-api-access-wrvs9\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050184    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050200    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050213    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050223    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xnwd\" (UniqueName: \"kubernetes.io/projected/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-kube-api-access-9xnwd\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050232    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050242    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050255    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4257a9b-63b8-4ca3-896a-080c7fb3853f-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050269    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.050364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.050284    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4257a9b-63b8-4ca3-896a-080c7fb3853f-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.301173 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.297058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:14.328482 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.328185    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:45:14.368957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.368919    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.369123 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.368978    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.369394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.369370    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.369604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.369566    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:14.369763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.369742    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.369876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.369856    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:14.370052 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.370034    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.370132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.370072    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.370193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.370174    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.370282 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.370253    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.370838 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.370548    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:14.371263 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.371222    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:14.373679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.373653    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.373814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.373684    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.373814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.373700    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.373814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.373715    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.377070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.376955    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:45:14.377070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.376958    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:14.377210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.377120    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:14.377264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.377222    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "kube-api-access-pwvms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:14.474670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.474634    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert\") pod \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\" (UID: \"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1\") "
Nov 01 22:45:14.475022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.474990    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert" pod "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" (UID: "3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:14.475636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.475617    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.475636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.475637    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwvms\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-kube-api-access-pwvms\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.475818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.475647    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.475818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.475657    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.475818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.475667    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:14.645550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.645516    5057 generic.go:356] "Generic (PLEG): container finished" podID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerID="4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0" exitCode=0
Nov 01 22:45:14.645748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.645563    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" event={"ID":"d4257a9b-63b8-4ca3-896a-080c7fb3853f","Type":"ContainerDied","Data":"4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0"}
Nov 01 22:45:14.645748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.645588    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"
Nov 01 22:45:14.645748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.645635    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt" event={"ID":"d4257a9b-63b8-4ca3-896a-080c7fb3853f","Type":"ContainerDied","Data":"67e3181d0851103446172e15468a7d5e230db3b2773909d3df97eecf18a42567"}
Nov 01 22:45:14.645748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.645675    5057 scope.go:122] "RemoveContainer" containerID="4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0"
Nov 01 22:45:14.654354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.654324    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5" event={"ID":"3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1","Type":"ContainerDied","Data":"b6d3e54f27321d931d3685167f12a64a2ad5bd35da14457d9d360e3b3b10388f"}
Nov 01 22:45:14.654595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.654415    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"
Nov 01 22:45:14.656885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.656856    5057 generic.go:356] "Generic (PLEG): container finished" podID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerID="bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8" exitCode=0
Nov 01 22:45:14.657014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.656998    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" event={"ID":"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5","Type":"ContainerDied","Data":"bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8"}
Nov 01 22:45:14.657076 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.657048    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd" event={"ID":"f64b1efe-a50d-49d7-a4bb-0a98e2d052f5","Type":"ContainerDied","Data":"ab373175abd37833e556a9fbe70c5b06bd5300c72c3bf6acd1dd6b7c044bc49c"}
Nov 01 22:45:14.657122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.657113    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"
Nov 01 22:45:14.673107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.672789    5057 scope.go:122] "RemoveContainer" containerID="4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0"
Nov 01 22:45:14.673180 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:14.673135    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0\": container with ID starting with 4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0 not found: ID does not exist" containerID="4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0"
Nov 01 22:45:14.673180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.673161    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0"} err="failed to get container status \"4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0\": rpc error: code = NotFound desc = could not find container \"4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0\": container with ID starting with 4881fb0d16a8011e3d39ff30dea421ac93a9e9b2139e80c31a5661d3161223b0 not found: ID does not exist"
Nov 01 22:45:14.673180 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.673177    5057 scope.go:122] "RemoveContainer" containerID="1c6b7db0b161849223c1daa9e2e541840c3f55bd86bb97ce6dd5481fb58b1d8e"
Nov 01 22:45:14.684717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.684684    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"]
Nov 01 22:45:14.686484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.686462    5057 scope.go:122] "RemoveContainer" containerID="bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8"
Nov 01 22:45:14.689350 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.689326    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-bxpdt"]
Nov 01 22:45:14.697890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.697868    5057 scope.go:122] "RemoveContainer" containerID="bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8"
Nov 01 22:45:14.698395 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:14.698350    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8\": container with ID starting with bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8 not found: ID does not exist" containerID="bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8"
Nov 01 22:45:14.698488 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.698407    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8"} err="failed to get container status \"bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8\": rpc error: code = NotFound desc = could not find container \"bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8\": container with ID starting with bb7d4d37323e05f25e53bfbfd9764592d3b4c4c622624756bb1f655bf612d9f8 not found: ID does not exist"
Nov 01 22:45:14.709485 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.709454    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"]
Nov 01 22:45:14.715517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.715476    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-jvqk5"]
Nov 01 22:45:14.732794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.732765    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"]
Nov 01 22:45:14.740249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:14.740219    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-e25182-openshift-default-597bd67b47-6w4fd"]
Nov 01 22:45:15.254749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:15.254717    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:15.406428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:15.406391    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" path="/var/lib/kubelet/pods/3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1/volumes"
Nov 01 22:45:15.407053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:15.407031    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" path="/var/lib/kubelet/pods/d4257a9b-63b8-4ca3-896a-080c7fb3853f/volumes"
Nov 01 22:45:15.407605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:15.407580    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" path="/var/lib/kubelet/pods/f64b1efe-a50d-49d7-a4bb-0a98e2d052f5/volumes"
Nov 01 22:45:16.111844 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:16.111761    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:17.014640 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:17.013721    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:17.888302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:17.888269    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:18.075151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:18.075115    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:18.435515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:18.432267    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:18.970443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:18.970403    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"]
Nov 01 22:45:18.970695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:18.970670    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy" containerID="cri-o://fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145" gracePeriod=30
Nov 01 22:45:19.284272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.284184    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main" containerID="cri-o://fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03" gracePeriod=45
Nov 01 22:45:19.640991 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.640950    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:45:19.691240 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.691204    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm\") pod \"858b7e22-cc93-4c35-b03f-fd8c833ede79\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") "
Nov 01 22:45:19.691422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.691285    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home\") pod \"858b7e22-cc93-4c35-b03f-fd8c833ede79\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") "
Nov 01 22:45:19.691422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.691317    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d\" (UniqueName: \"kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d\") pod \"858b7e22-cc93-4c35-b03f-fd8c833ede79\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") "
Nov 01 22:45:19.691422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.691342    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir\") pod \"858b7e22-cc93-4c35-b03f-fd8c833ede79\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") "
Nov 01 22:45:19.691422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.691391    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache\") pod \"858b7e22-cc93-4c35-b03f-fd8c833ede79\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") "
Nov 01 22:45:19.691687 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.691423    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs\" (UniqueName: \"kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs\") pod \"858b7e22-cc93-4c35-b03f-fd8c833ede79\" (UID: \"858b7e22-cc93-4c35-b03f-fd8c833ede79\") "
Nov 01 22:45:19.697644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.695862    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir" pod "858b7e22-cc93-4c35-b03f-fd8c833ede79" (UID: "858b7e22-cc93-4c35-b03f-fd8c833ede79"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:19.697644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.695959    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home" pod "858b7e22-cc93-4c35-b03f-fd8c833ede79" (UID: "858b7e22-cc93-4c35-b03f-fd8c833ede79"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:19.697644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.696074    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache" pod "858b7e22-cc93-4c35-b03f-fd8c833ede79" (UID: "858b7e22-cc93-4c35-b03f-fd8c833ede79"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:19.715043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.715004    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm" pod "858b7e22-cc93-4c35-b03f-fd8c833ede79" (UID: "858b7e22-cc93-4c35-b03f-fd8c833ede79"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:19.716613 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.715800    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d" pod "858b7e22-cc93-4c35-b03f-fd8c833ede79" (UID: "858b7e22-cc93-4c35-b03f-fd8c833ede79"). InnerVolumeSpecName "kube-api-access-zhl9d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:19.732601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.724089    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs" pod "858b7e22-cc93-4c35-b03f-fd8c833ede79" (UID: "858b7e22-cc93-4c35-b03f-fd8c833ede79"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:19.812148 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.810681    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:19.834534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.833680    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:19.834534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.833716    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhl9d\" (UniqueName: \"kubernetes.io/projected/858b7e22-cc93-4c35-b03f-fd8c833ede79-kube-api-access-zhl9d\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:19.834534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.833735    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:19.834534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.833750    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/858b7e22-cc93-4c35-b03f-fd8c833ede79-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:19.834534 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.833767    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/858b7e22-cc93-4c35-b03f-fd8c833ede79-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:19.867083 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.866988    5057 generic.go:356] "Generic (PLEG): container finished" podID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerID="fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03" exitCode=0
Nov 01 22:45:19.867311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.867289    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" event={"ID":"858b7e22-cc93-4c35-b03f-fd8c833ede79","Type":"ContainerDied","Data":"fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03"}
Nov 01 22:45:19.867422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.867409    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx" event={"ID":"858b7e22-cc93-4c35-b03f-fd8c833ede79","Type":"ContainerDied","Data":"53a2c4884d02205fa3e098949306a71bcf15edf5ba95c34f6a2ba2cdd2d5074d"}
Nov 01 22:45:19.867539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.867512    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"
Nov 01 22:45:19.867693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.867677    5057 scope.go:122] "RemoveContainer" containerID="fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03"
Nov 01 22:45:19.889237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.889213    5057 scope.go:122] "RemoveContainer" containerID="fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03"
Nov 01 22:45:19.889678 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:19.889645    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03\": container with ID starting with fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03 not found: ID does not exist" containerID="fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03"
Nov 01 22:45:19.889794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.889690    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03"} err="failed to get container status \"fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03\": rpc error: code = NotFound desc = could not find container \"fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03\": container with ID starting with fc10180633cca323a22fd828c9df395ef8d88d322e6eb3f5af610cb029b29b03 not found: ID does not exist"
Nov 01 22:45:19.902540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.902471    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"]
Nov 01 22:45:19.910671 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:19.909678    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b42cc4/distinct-2-w4-b42cc4-kserve-5b8665858d-lt6mx"]
Nov 01 22:45:20.390343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:20.390303    5057 scope.go:122] "RemoveContainer" containerID="ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393"
Nov 01 22:45:20.895816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:20.895779    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:21.061298 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.061179    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:21.279659 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.279598    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:21.398438 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.398364    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" path="/var/lib/kubelet/pods/858b7e22-cc93-4c35-b03f-fd8c833ede79/volumes"
Nov 01 22:45:21.886810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.886775    5057 generic.go:356] "Generic (PLEG): container finished" podID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerID="c8f12ce4fa13e89518afcf6eef8dc6c4ea09ddb6afa23ccd9f2b3eb022b3de97" exitCode=60
Nov 01 22:45:21.887004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.886824    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerDied","Data":"c8f12ce4fa13e89518afcf6eef8dc6c4ea09ddb6afa23ccd9f2b3eb022b3de97"}
Nov 01 22:45:21.887004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.886873    5057 scope.go:122] "RemoveContainer" containerID="ab92bf6f9c0ba6c4e11fa76e1e17e065bad92d998f066f5631853aa27a854393"
Nov 01 22:45:21.887309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:21.887289    5057 scope.go:122] "RemoveContainer" containerID="c8f12ce4fa13e89518afcf6eef8dc6c4ea09ddb6afa23ccd9f2b3eb022b3de97"
Nov 01 22:45:21.887684 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:21.887658    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds_odh-ai-gateway-infra(0305b796-0ef6-47ef-9c93-2359677f72f8)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8"
Nov 01 22:45:22.390189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:22.390154    5057 scope.go:122] "RemoveContainer" containerID="f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2"
Nov 01 22:45:22.396909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:22.396874    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:22.895914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:22.895876    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerStarted","Data":"db6ea6b4db0a07581c62378b99f2498a266d3693f571a3a287e0ccf968faa28f"}
Nov 01 22:45:22.929922 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:22.929885    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"]
Nov 01 22:45:23.135449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.135424    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:23.208547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.208461    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222\" (UniqueName: \"kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222\") pod \"0305b796-0ef6-47ef-9c93-2359677f72f8\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") "
Nov 01 22:45:23.208753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.208637    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca\") pod \"0305b796-0ef6-47ef-9c93-2359677f72f8\" (UID: \"0305b796-0ef6-47ef-9c93-2359677f72f8\") "
Nov 01 22:45:23.209088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.209060    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca" pod "0305b796-0ef6-47ef-9c93-2359677f72f8" (UID: "0305b796-0ef6-47ef-9c93-2359677f72f8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:23.209614 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.209593    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0305b796-0ef6-47ef-9c93-2359677f72f8-maas-api-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:23.211146 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.211117    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222" pod "0305b796-0ef6-47ef-9c93-2359677f72f8" (UID: "0305b796-0ef6-47ef-9c93-2359677f72f8"). InnerVolumeSpecName "kube-api-access-ll222". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:23.311012 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.310929    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ll222\" (UniqueName: \"kubernetes.io/projected/0305b796-0ef6-47ef-9c93-2359677f72f8-kube-api-access-ll222\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:23.362760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.362722    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:23.905665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.905634    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"
Nov 01 22:45:23.906154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.905665    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds" event={"ID":"0305b796-0ef6-47ef-9c93-2359677f72f8","Type":"ContainerDied","Data":"d35ba31e9e64f247bb8b8d1780dafdc4c0833991edffbc578e98ee3c3beed4b3"}
Nov 01 22:45:23.906154 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.905736    5057 scope.go:122] "RemoveContainer" containerID="c8f12ce4fa13e89518afcf6eef8dc6c4ea09ddb6afa23ccd9f2b3eb022b3de97"
Nov 01 22:45:23.908132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.908081    5057 generic.go:356] "Generic (PLEG): container finished" podID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerID="db6ea6b4db0a07581c62378b99f2498a266d3693f571a3a287e0ccf968faa28f" exitCode=60
Nov 01 22:45:23.908292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.908125    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerDied","Data":"db6ea6b4db0a07581c62378b99f2498a266d3693f571a3a287e0ccf968faa28f"}
Nov 01 22:45:23.908657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.908629    5057 scope.go:122] "RemoveContainer" containerID="db6ea6b4db0a07581c62378b99f2498a266d3693f571a3a287e0ccf968faa28f"
Nov 01 22:45:23.909145 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:23.909117    5057 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd_odh-ai-gateway-infra(86664679-f7d5-471e-b41b-98cdb6f48fdd)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd"
Nov 01 22:45:23.916959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.916936    5057 scope.go:122] "RemoveContainer" containerID="f5b8d9904d38fa128cf77a28a309b3459a2aa3a51c8952471b40ba6123f208c2"
Nov 01 22:45:23.968697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.968667    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"]
Nov 01 22:45:23.985910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.982542    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"]
Nov 01 22:45:23.987632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:23.987563    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-10a1d9-29892885-27pds"]
Nov 01 22:45:24.172064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.171715    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:24.317155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.311249    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:24.368033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.367876    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:45:24.471241 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471205    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471257    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471332    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471372    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471401    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471525    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471557    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471623    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:24.471702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471644    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471775    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:24.471894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471789    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token\") pod \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\" (UID: \"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf\") "
Nov 01 22:45:24.471894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471790    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:24.472015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471887    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:24.472015 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.471947    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:24.472633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.472606    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.472633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.472634    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.472814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.472651    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.472814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.472665    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.472814 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.472679    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.473962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.473936    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:45:24.474063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.474023    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:45:24.474063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.474047    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:24.474128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.474081    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd" pod "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" (UID: "f842a24a-6ba4-46b9-a3e4-7a5c624a27bf"). InnerVolumeSpecName "kube-api-access-mrlmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:24.573713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.573631    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.573713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.573663    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.573713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.573674    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.573713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.573683    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrlmd\" (UniqueName: \"kubernetes.io/projected/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf-kube-api-access-mrlmd\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:24.969431 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.969398    5057 generic.go:356] "Generic (PLEG): container finished" podID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerID="fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145" exitCode=0
Nov 01 22:45:24.970108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.970082    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" event={"ID":"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf","Type":"ContainerDied","Data":"fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145"}
Nov 01 22:45:24.970288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.970263    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr" event={"ID":"f842a24a-6ba4-46b9-a3e4-7a5c624a27bf","Type":"ContainerDied","Data":"3cfb60cd12878393b4d2ca2bd86b353d198ab78659a7778a5ffed17a78909af1"}
Nov 01 22:45:24.970421 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.970407    5057 scope.go:122] "RemoveContainer" containerID="fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145"
Nov 01 22:45:24.970690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:24.970676    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"
Nov 01 22:45:25.036694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.035697    5057 scope.go:122] "RemoveContainer" containerID="c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"
Nov 01 22:45:25.056852 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.056541    5057 scope.go:122] "RemoveContainer" containerID="fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145"
Nov 01 22:45:25.057004 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:25.056945    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145\": container with ID starting with fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145 not found: ID does not exist" containerID="fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145"
Nov 01 22:45:25.057004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.056992    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145"} err="failed to get container status \"fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145\": rpc error: code = NotFound desc = could not find container \"fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145\": container with ID starting with fcc1998a553b07fcc212cb752b8ee11c671cabb87d8805e74e8aa7d6f0f5b145 not found: ID does not exist"
Nov 01 22:45:25.057143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.057011    5057 scope.go:122] "RemoveContainer" containerID="c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"
Nov 01 22:45:25.057258 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:25.057238    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0\": container with ID starting with c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0 not found: ID does not exist" containerID="c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"
Nov 01 22:45:25.057321 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.057267    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0"} err="failed to get container status \"c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0\": rpc error: code = NotFound desc = could not find container \"c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0\": container with ID starting with c64e035b544612bd4a163493bf67e3901a661213d69f7af59bd6520f8be01db0 not found: ID does not exist"
Nov 01 22:45:25.068626 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.066060    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"]
Nov 01 22:45:25.074581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.072551    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-425870c1-openshift-default-65d9486fdf-pplbr"]
Nov 01 22:45:25.269416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.269387    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:25.317213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.317075    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:25.403706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.403620    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" path="/var/lib/kubelet/pods/0305b796-0ef6-47ef-9c93-2359677f72f8/volumes"
Nov 01 22:45:25.404466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.404433    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" path="/var/lib/kubelet/pods/f842a24a-6ba4-46b9-a3e4-7a5c624a27bf/volumes"
Nov 01 22:45:25.446961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.446924    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2\" (UniqueName: \"kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2\") pod \"86664679-f7d5-471e-b41b-98cdb6f48fdd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") "
Nov 01 22:45:25.447132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.447065    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca\") pod \"86664679-f7d5-471e-b41b-98cdb6f48fdd\" (UID: \"86664679-f7d5-471e-b41b-98cdb6f48fdd\") "
Nov 01 22:45:25.447538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.447506    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca" pod "86664679-f7d5-471e-b41b-98cdb6f48fdd" (UID: "86664679-f7d5-471e-b41b-98cdb6f48fdd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:25.447711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.447690    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86664679-f7d5-471e-b41b-98cdb6f48fdd-maas-api-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:25.449536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.449486    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2" pod "86664679-f7d5-471e-b41b-98cdb6f48fdd" (UID: "86664679-f7d5-471e-b41b-98cdb6f48fdd"). InnerVolumeSpecName "kube-api-access-qv8c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:25.549322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.549233    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qv8c2\" (UniqueName: \"kubernetes.io/projected/86664679-f7d5-471e-b41b-98cdb6f48fdd-kube-api-access-qv8c2\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:25.999712 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.999677    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"
Nov 01 22:45:26.000211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.999676    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd" event={"ID":"86664679-f7d5-471e-b41b-98cdb6f48fdd","Type":"ContainerDied","Data":"e7f0800126524d41e66866cb50f51d59a0195df95b8d9cad409692839ab19f47"}
Nov 01 22:45:26.000211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:25.999823    5057 scope.go:122] "RemoveContainer" containerID="db6ea6b4db0a07581c62378b99f2498a266d3693f571a3a287e0ccf968faa28f"
Nov 01 22:45:26.026484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:26.026452    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"]
Nov 01 22:45:26.031043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:26.031012    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-8f2555-29892885-b2dkd"]
Nov 01 22:45:26.209731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:26.209692    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:27.132193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:27.132155    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:27.399589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:27.399482    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" path="/var/lib/kubelet/pods/86664679-f7d5-471e-b41b-98cdb6f48fdd/volumes"
Nov 01 22:45:27.797296 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:27.797263    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:27.938515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:27.926012    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:28.092155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:28.086837    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:28.398387 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:28.398289    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:28.817110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:28.817062    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:29.129754 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.129669    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"]
Nov 01 22:45:29.130193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.130156    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x" podUID="848cd2d7-99cb-499a-b13a-19782bfed719" containerName="maas-api" containerID="cri-o://63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4" gracePeriod=30
Nov 01 22:45:29.249647 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.247370    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:29.403816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.403729    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:29.711518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.711163    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:45:29.787914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.787821    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca\") pod \"848cd2d7-99cb-499a-b13a-19782bfed719\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") "
Nov 01 22:45:29.787914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.787882    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls\") pod \"848cd2d7-99cb-499a-b13a-19782bfed719\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") "
Nov 01 22:45:29.788287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.788073    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv\" (UniqueName: \"kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv\") pod \"848cd2d7-99cb-499a-b13a-19782bfed719\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") "
Nov 01 22:45:29.788287 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.788119    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs\") pod \"848cd2d7-99cb-499a-b13a-19782bfed719\" (UID: \"848cd2d7-99cb-499a-b13a-19782bfed719\") "
Nov 01 22:45:29.788601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.788573    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca" pod "848cd2d7-99cb-499a-b13a-19782bfed719" (UID: "848cd2d7-99cb-499a-b13a-19782bfed719"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:29.788824 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.788767    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/848cd2d7-99cb-499a-b13a-19782bfed719-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:29.791529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.790669    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls" pod "848cd2d7-99cb-499a-b13a-19782bfed719" (UID: "848cd2d7-99cb-499a-b13a-19782bfed719"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:29.791529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.790935    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv" pod "848cd2d7-99cb-499a-b13a-19782bfed719" (UID: "848cd2d7-99cb-499a-b13a-19782bfed719"). InnerVolumeSpecName "kube-api-access-wqdzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:29.791672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.791638    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs" pod "848cd2d7-99cb-499a-b13a-19782bfed719" (UID: "848cd2d7-99cb-499a-b13a-19782bfed719"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:29.890645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.890607    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqdzv\" (UniqueName: \"kubernetes.io/projected/848cd2d7-99cb-499a-b13a-19782bfed719-kube-api-access-wqdzv\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:29.890645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.890646    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:29.890830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.890661    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/848cd2d7-99cb-499a-b13a-19782bfed719-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:29.977201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:29.977148    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:30.087886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.087848    5057 generic.go:356] "Generic (PLEG): container finished" podID="848cd2d7-99cb-499a-b13a-19782bfed719" containerID="63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4" exitCode=0
Nov 01 22:45:30.088038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.087990    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x" event={"ID":"848cd2d7-99cb-499a-b13a-19782bfed719","Type":"ContainerDied","Data":"63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4"}
Nov 01 22:45:30.088126 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.088040    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x" event={"ID":"848cd2d7-99cb-499a-b13a-19782bfed719","Type":"ContainerDied","Data":"9b1bbbdfea11368997f889961f68f9c00b689bace4525b25dc796cd4c4a1e189"}
Nov 01 22:45:30.088126 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.088069    5057 scope.go:122] "RemoveContainer" containerID="63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4"
Nov 01 22:45:30.088245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.088210    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"
Nov 01 22:45:30.103542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.103478    5057 scope.go:122] "RemoveContainer" containerID="63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4"
Nov 01 22:45:30.103874 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:30.103850    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4\": container with ID starting with 63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4 not found: ID does not exist" containerID="63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4"
Nov 01 22:45:30.103935 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.103882    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4"} err="failed to get container status \"63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4\": rpc error: code = NotFound desc = could not find container \"63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4\": container with ID starting with 63c8cbf04ad9e560737d566dc5ce483bc63f66d1891780c9ae152e93c1d004b4 not found: ID does not exist"
Nov 01 22:45:30.113166 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.113139    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:30.124229 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.124188    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"]
Nov 01 22:45:30.128281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.128249    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-10a1d9-7d748b554c-dlt9x"]
Nov 01 22:45:30.511475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.511441    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:30.965210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:30.965163    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:31.399608 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:31.399520    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="848cd2d7-99cb-499a-b13a-19782bfed719" path="/var/lib/kubelet/pods/848cd2d7-99cb-499a-b13a-19782bfed719/volumes"
Nov 01 22:45:31.948529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:31.948474    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:32.111798 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:32.111746    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:32.901341 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:32.901247    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:33.877644 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:33.877606    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:34.706737 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:34.706700    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:35.365573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:35.365466    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:36.397226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:36.397189    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:36.575411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:36.575376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:36.717132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:36.717098    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:36.915516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:36.914389    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:37.987957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:37.987922    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:39.011738 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:39.011679    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:39.180976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:39.180945    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:39.358057 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:39.357959    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:39.516572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:39.516534    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:40.016975 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:40.016930    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:45:40.282870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:40.282778    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:40.992865 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:40.992830    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:41.418361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:41.418274    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:41.944558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:41.908168    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:42.210963 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:42.210184    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:42.368522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:42.365453    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:42.564074 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:42.563948    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:43.973841 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:43.973789    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:44.270522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:44.270423    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:44.453908 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:44.453873    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:44.793048 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:44.793015    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:45.208174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:45.208138    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:45.388539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:45.386852    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:45.719802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:45.719766    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:45.922924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:45.922881    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:46.195365 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:46.195257    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:47.229060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.228996    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:47.452455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.452420    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"]
Nov 01 22:45:47.452701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.452674    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4" podUID="96be7af1-f3bf-4aa2-82c7-24d84f474120" containerName="maas-api" containerID="cri-o://e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae" gracePeriod=30
Nov 01 22:45:47.600736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.600695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:47.697710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.687615    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"]
Nov 01 22:45:47.859796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.859716    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:45:47.898518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.898219    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls\") pod \"96be7af1-f3bf-4aa2-82c7-24d84f474120\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") "
Nov 01 22:45:47.898518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.898291    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p\" (UniqueName: \"kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p\") pod \"96be7af1-f3bf-4aa2-82c7-24d84f474120\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") "
Nov 01 22:45:47.898518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.898436    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca\") pod \"96be7af1-f3bf-4aa2-82c7-24d84f474120\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") "
Nov 01 22:45:47.898916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.898603    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs\") pod \"96be7af1-f3bf-4aa2-82c7-24d84f474120\" (UID: \"96be7af1-f3bf-4aa2-82c7-24d84f474120\") "
Nov 01 22:45:47.921519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.919366    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca" pod "96be7af1-f3bf-4aa2-82c7-24d84f474120" (UID: "96be7af1-f3bf-4aa2-82c7-24d84f474120"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:47.993869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:47.981105    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs" pod "96be7af1-f3bf-4aa2-82c7-24d84f474120" (UID: "96be7af1-f3bf-4aa2-82c7-24d84f474120"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:48.011515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.002719    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/96be7af1-f3bf-4aa2-82c7-24d84f474120-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:48.011515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.002761    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:48.011515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.004234    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls" pod "96be7af1-f3bf-4aa2-82c7-24d84f474120" (UID: "96be7af1-f3bf-4aa2-82c7-24d84f474120"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:45:48.024728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.015409    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p" pod "96be7af1-f3bf-4aa2-82c7-24d84f474120" (UID: "96be7af1-f3bf-4aa2-82c7-24d84f474120"). InnerVolumeSpecName "kube-api-access-nkt4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:48.057527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.053173    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:48.140572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.135342    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/96be7af1-f3bf-4aa2-82c7-24d84f474120-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:48.140572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.135383    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkt4p\" (UniqueName: \"kubernetes.io/projected/96be7af1-f3bf-4aa2-82c7-24d84f474120-kube-api-access-nkt4p\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:48.334045 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.334009    5057 generic.go:356] "Generic (PLEG): container finished" podID="96be7af1-f3bf-4aa2-82c7-24d84f474120" containerID="e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae" exitCode=0
Nov 01 22:45:48.334517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.334059    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4" event={"ID":"96be7af1-f3bf-4aa2-82c7-24d84f474120","Type":"ContainerDied","Data":"e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae"}
Nov 01 22:45:48.334517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.334069    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"
Nov 01 22:45:48.334517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.334105    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4" event={"ID":"96be7af1-f3bf-4aa2-82c7-24d84f474120","Type":"ContainerDied","Data":"3a8a95f27fdb8de082954b4252c7d6337517be24e67f4cd05271b2d0615dbbb4"}
Nov 01 22:45:48.334517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.334136    5057 scope.go:122] "RemoveContainer" containerID="e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae"
Nov 01 22:45:48.348121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.348094    5057 scope.go:122] "RemoveContainer" containerID="e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae"
Nov 01 22:45:48.348551 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:48.348472    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae\": container with ID starting with e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae not found: ID does not exist" containerID="e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae"
Nov 01 22:45:48.348656 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.348562    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae"} err="failed to get container status \"e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae\": rpc error: code = NotFound desc = could not find container \"e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae\": container with ID starting with e5b15ba4ed6100c8504ee42d71e80366838accfcd61af9c38988c9d4fa2860ae not found: ID does not exist"
Nov 01 22:45:48.362547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.362504    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:48.388523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.388023    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"]
Nov 01 22:45:48.412571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.411853    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-a9c08e-5b5dbff8dc-v87b4"]
Nov 01 22:45:48.645553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:48.622799    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:49.153600 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:49.152000    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:49.474645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:49.470421    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="96be7af1-f3bf-4aa2-82c7-24d84f474120" path="/var/lib/kubelet/pods/96be7af1-f3bf-4aa2-82c7-24d84f474120/volumes"
Nov 01 22:45:49.493526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:49.477361    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:50.128517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:50.128245    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:50.431249 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:50.431162    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:50.667861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:50.667829    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:51.191513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:51.163641    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:51.478525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:51.477789    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:52.098518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:52.089199    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:52.428576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:52.427091    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:52.689869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:52.689767    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4" podUID="6254f311-ea06-458c-abf6-b057560c94ba" containerName="payload-pre-processing" containerID="cri-o://2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870" gracePeriod=25
Nov 01 22:45:53.163749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.163721    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:45:53.257377 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.257327    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb\" (UniqueName: \"kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb\") pod \"6254f311-ea06-458c-abf6-b057560c94ba\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") "
Nov 01 22:45:53.257763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.257741    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume\") pod \"6254f311-ea06-458c-abf6-b057560c94ba\" (UID: \"6254f311-ea06-458c-abf6-b057560c94ba\") "
Nov 01 22:45:53.258163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.258141    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume" pod "6254f311-ea06-458c-abf6-b057560c94ba" (UID: "6254f311-ea06-458c-abf6-b057560c94ba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:45:53.259898 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.259867    5057 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6254f311-ea06-458c-abf6-b057560c94ba-plugins-config-volume\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:53.262004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.261933    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb" pod "6254f311-ea06-458c-abf6-b057560c94ba" (UID: "6254f311-ea06-458c-abf6-b057560c94ba"). InnerVolumeSpecName "kube-api-access-gb2sb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:45:53.353328 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.353290    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:53.361227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.361179    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb2sb\" (UniqueName: \"kubernetes.io/projected/6254f311-ea06-458c-abf6-b057560c94ba-kube-api-access-gb2sb\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:45:53.539995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.539903    5057 generic.go:356] "Generic (PLEG): container finished" podID="6254f311-ea06-458c-abf6-b057560c94ba" containerID="2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870" exitCode=0
Nov 01 22:45:53.539995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.539957    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"
Nov 01 22:45:53.539995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.539983    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4" event={"ID":"6254f311-ea06-458c-abf6-b057560c94ba","Type":"ContainerDied","Data":"2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870"}
Nov 01 22:45:53.540284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.540049    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4" event={"ID":"6254f311-ea06-458c-abf6-b057560c94ba","Type":"ContainerDied","Data":"ee875c7dd44f569a22ff0e80b27a4eee0b0a3d9ee5afd50f2ccaeee45951bb1b"}
Nov 01 22:45:53.540284 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.540082    5057 scope.go:122] "RemoveContainer" containerID="2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870"
Nov 01 22:45:53.552338 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.552307    5057 scope.go:122] "RemoveContainer" containerID="2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870"
Nov 01 22:45:53.552778 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:45:53.552734    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870\": container with ID starting with 2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870 not found: ID does not exist" containerID="2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870"
Nov 01 22:45:53.552886 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.552790    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870"} err="failed to get container status \"2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870\": rpc error: code = NotFound desc = could not find container \"2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870\": container with ID starting with 2a293571ea587131a8e5aef8379e204fe30f92c555d25a339729756d54858870 not found: ID does not exist"
Nov 01 22:45:53.582510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.582448    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"]
Nov 01 22:45:53.586027 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:53.586003    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-8f2555-5fd587b46c-hcbv4"]
Nov 01 22:45:54.461947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:54.461592    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:55.398178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:55.398137    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6254f311-ea06-458c-abf6-b057560c94ba" path="/var/lib/kubelet/pods/6254f311-ea06-458c-abf6-b057560c94ba/volumes"
Nov 01 22:45:55.637123 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:55.637053    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:55.929978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:55.929943    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:56.815026 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:56.814982    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:58.004732 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.004620    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:58.296044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.295951    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:58.590584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.589225    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:58.627632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.627598    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"]
Nov 01 22:45:58.627836 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.627814    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy" containerID="cri-o://e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437" gracePeriod=30
Nov 01 22:45:58.658318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.658286    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"]
Nov 01 22:45:58.658566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.658539    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" containerID="cri-o://715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947" gracePeriod=30
Nov 01 22:45:58.831516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:58.830535    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:59.530684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.530647    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:45:59.554239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554203    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"]
Nov 01 22:45:59.554880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554861    5057 cpu_manager.go:414] "containerMap: removing container" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy"
Nov 01 22:45:59.554880 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554880    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554890    5057 cpu_manager.go:414] "containerMap: removing container" podUID="6254f311-ea06-458c-abf6-b057560c94ba" containerName="payload-pre-processing"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554896    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6254f311-ea06-458c-abf6-b057560c94ba" containerName="payload-pre-processing"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554919    5057 cpu_manager.go:414] "containerMap: removing container" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554924    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554932    5057 cpu_manager.go:414] "containerMap: removing container" podUID="cd8aa617-b822-4f46-8bb2-6bdb371b9982" containerName="maas-api"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554939    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cd8aa617-b822-4f46-8bb2-6bdb371b9982" containerName="maas-api"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554948    5057 cpu_manager.go:414] "containerMap: removing container" podUID="848cd2d7-99cb-499a-b13a-19782bfed719" containerName="maas-api"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554953    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="848cd2d7-99cb-499a-b13a-19782bfed719" containerName="maas-api"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554959    5057 cpu_manager.go:414] "containerMap: removing container" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554964    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554980    5057 cpu_manager.go:414] "containerMap: removing container" podUID="96be7af1-f3bf-4aa2-82c7-24d84f474120" containerName="maas-api"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554984    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="96be7af1-f3bf-4aa2-82c7-24d84f474120" containerName="maas-api"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554994    5057 cpu_manager.go:414] "containerMap: removing container" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.554999    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555005    5057 cpu_manager.go:414] "containerMap: removing container" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555010    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555018    5057 cpu_manager.go:414] "containerMap: removing container" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555023    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555036    5057 cpu_manager.go:414] "containerMap: removing container" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555040    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555048    5057 cpu_manager.go:414] "containerMap: removing container" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555053    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555062    5057 cpu_manager.go:414] "containerMap: removing container" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555067    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555075    5057 cpu_manager.go:414] "containerMap: removing container" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main"
Nov 01 22:45:59.555061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555080    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555098    5057 cpu_manager.go:414] "containerMap: removing container" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555102    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555195    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555206    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4257a9b-63b8-4ca3-896a-080c7fb3853f" containerName="istio-proxy"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555212    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="6254f311-ea06-458c-abf6-b057560c94ba" containerName="payload-pre-processing"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555219    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555224    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="cd8aa617-b822-4f46-8bb2-6bdb371b9982" containerName="maas-api"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555230    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="848cd2d7-99cb-499a-b13a-19782bfed719" containerName="maas-api"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555234    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555241    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="96be7af1-f3bf-4aa2-82c7-24d84f474120" containerName="maas-api"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555247    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="f64b1efe-a50d-49d7-a4bb-0a98e2d052f5" containerName="istio-proxy"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555253    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="f842a24a-6ba4-46b9-a3e4-7a5c624a27bf" containerName="istio-proxy"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555257    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555264    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c8c0f55-2c7d-45fc-85ff-ac07c88f07e1" containerName="istio-proxy"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555270    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="0305b796-0ef6-47ef-9c93-2359677f72f8" containerName="cleanup"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555277    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="858b7e22-cc93-4c35-b03f-fd8c833ede79" containerName="main"
Nov 01 22:45:59.556370 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.555282    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:45:59.560320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.560299    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.570659 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.570638    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1fddbd-openshift-default-dockercfg-5zxzj\""
Nov 01 22:45:59.587051 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.587024    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"]
Nov 01 22:45:59.667420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667378    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.667616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667431    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.667616 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667508    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.667771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667679    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v56kn\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.667771 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667746    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.667895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667783    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.667895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667844    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.668009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667896    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.668009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.667921    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.768859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.768823    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.768898    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.768946    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.768973    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769075    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769111    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769168    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769292    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v56kn\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769348    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769473 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769428    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769508    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769760    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769828    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.769864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.769845    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.771730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.771706    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.771941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.771922    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.778766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.778740    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v56kn\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.779926 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.779907    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:45:59.874612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:45:59.874540    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:46:00.168637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.168614    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"]
Nov 01 22:46:00.170851 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:46:00.170820    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafbfc2e3_7c03_40b2_a82b_be84c738f252.slice/crio-1cb79489385daa0dde9a96bd43b770bf07e5b591d61216197a9702df59c4037d WatchSource:0}: Error finding container 1cb79489385daa0dde9a96bd43b770bf07e5b591d61216197a9702df59c4037d: Status 404 returned error can't find the container with id 1cb79489385daa0dde9a96bd43b770bf07e5b591d61216197a9702df59c4037d
Nov 01 22:46:00.173761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.173724    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:46:00.173859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.173809    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:46:00.173859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.173840    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:46:00.539746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.539706    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:00.617044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.617006    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" event={"ID":"afbfc2e3-7c03-40b2-a82b-be84c738f252","Type":"ContainerStarted","Data":"41341b41d98777c9ed47d6338d9a7b4dca17ea533770924644e76cc589097c17"}
Nov 01 22:46:00.617285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.617069    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" event={"ID":"afbfc2e3-7c03-40b2-a82b-be84c738f252","Type":"ContainerStarted","Data":"1cb79489385daa0dde9a96bd43b770bf07e5b591d61216197a9702df59c4037d"}
Nov 01 22:46:00.659081 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.659030    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podStartSLOduration=1.659016218 podStartE2EDuration="1.659016218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:46:00.655635808 +0000 UTC m=+1417.397331426" watchObservedRunningTime="2026-11-01 22:46:00.659016218 +0000 UTC m=+1417.400711837"
Nov 01 22:46:00.711225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.711191    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:00.818596 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.816423    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"]
Nov 01 22:46:00.818596 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.816654    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" containerID="cri-o://114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b" gracePeriod=30
Nov 01 22:46:00.875927 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.875892    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:46:00.932566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:00.918608    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:01.680543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:01.678781    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"]
Nov 01 22:46:01.680543 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:01.679025    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4" podUID="2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" containerName="maas-api" containerID="cri-o://0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6" gracePeriod=30
Nov 01 22:46:01.745204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:01.745163    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:01.883821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:01.883423    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 22:46:01.883821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:01.883486    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 22:46:02.293432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.293365    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:02.309881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.309834    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:46:02.376113 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.376078    5057 patch_prober.go:29] interesting pod/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:46:02.376280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.376132    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:46:02.416828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.416771    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs\") pod \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") "
Nov 01 22:46:02.417019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.416884    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls\") pod \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") "
Nov 01 22:46:02.417019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.416972    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr\" (UniqueName: \"kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr\") pod \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") "
Nov 01 22:46:02.417144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.417037    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca\") pod \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\" (UID: \"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e\") "
Nov 01 22:46:02.417465 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.417430    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca" pod "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" (UID: "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:02.420538 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.417795    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:02.453615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.452548    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls" pod "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" (UID: "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:02.453615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.452551    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs" pod "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" (UID: "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:02.453615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.453123    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:02.454680 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.454656    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr" pod "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" (UID: "2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e"). InnerVolumeSpecName "kube-api-access-ch9xr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:02.559772 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.528482    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:02.560107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.560081    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:02.560247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.560233    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ch9xr\" (UniqueName: \"kubernetes.io/projected/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e-kube-api-access-ch9xr\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:02.659225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.657956    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:02.700484 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.700448    5057 generic.go:356] "Generic (PLEG): container finished" podID="2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" containerID="0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6" exitCode=0
Nov 01 22:46:02.700944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.700582    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"
Nov 01 22:46:02.700944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.700616    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4" event={"ID":"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e","Type":"ContainerDied","Data":"0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6"}
Nov 01 22:46:02.700944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.700683    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4" event={"ID":"2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e","Type":"ContainerDied","Data":"e0242e4f0851bf535b52abcd2a96f9676b75c52e21409e4efac5b6c9ef5e0def"}
Nov 01 22:46:02.700944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.700719    5057 scope.go:122] "RemoveContainer" containerID="0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6"
Nov 01 22:46:02.714780 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.714751    5057 scope.go:122] "RemoveContainer" containerID="0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6"
Nov 01 22:46:02.715321 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:02.715295    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6\": container with ID starting with 0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6 not found: ID does not exist" containerID="0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6"
Nov 01 22:46:02.715425 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.715331    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6"} err="failed to get container status \"0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6\": rpc error: code = NotFound desc = could not find container \"0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6\": container with ID starting with 0d8deb5e421d7bba17dfeae5dc7c819c2ea973f0a454217560f921773de4eed6 not found: ID does not exist"
Nov 01 22:46:02.826522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.826429    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:02.867514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.867236    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"]
Nov 01 22:46:02.885514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.878800    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 22:46:02.885514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.878884    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 22:46:02.909694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:02.906730    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-41b8f5-94ff8bb8b-n9gh4"]
Nov 01 22:46:03.220519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:03.216259    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:03.359606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:03.359570    5057 patch_prober.go:29] interesting pod/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:46:03.359869 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:03.359841    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:46:03.403893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:03.403824    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" path="/var/lib/kubelet/pods/2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e/volumes"
Nov 01 22:46:03.878882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:03.876856    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 22:46:03.878882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:03.876922    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 22:46:04.070439 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.053333    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:04.099121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.099088    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:46:04.188464 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.188442    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:46:04.281950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.281859    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.281950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.281920    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.281954    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.281993    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282108    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282178    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282288    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282418 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282334    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282508    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token\") pod \"fed73a15-8dd4-451b-8110-46a3f73c22c2\" (UID: \"fed73a15-8dd4-451b-8110-46a3f73c22c2\") "
Nov 01 22:46:04.282893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282862    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.283065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.282960    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:04.283351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.283325    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fed73a15-8dd4-451b-8110-46a3f73c22c2-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.283604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.283581    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.283703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.283623    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.283879 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.283856    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.284063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.283962    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.285859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.285837    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:04.286017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.285991    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "kube-api-access-gfhc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:04.286602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.286569    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.286882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.286858    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo" pod "fed73a15-8dd4-451b-8110-46a3f73c22c2" (UID: "fed73a15-8dd4-451b-8110-46a3f73c22c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:46:04.384332 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384299    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384549    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384584    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384655 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384631    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384669    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384698    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384714    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:04.384913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384806    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.384913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384838    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.385023 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.384996    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket\") pod \"b9ecdaf9-8526-430b-9373-ec36c6897253\" (UID: \"b9ecdaf9-8526-430b-9373-ec36c6897253\") "
Nov 01 22:46:04.385511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385467    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385548    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385567    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gfhc2\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-kube-api-access-gfhc2\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385581    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b9ecdaf9-8526-430b-9373-ec36c6897253-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385595    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385608    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385621    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fed73a15-8dd4-451b-8110-46a3f73c22c2-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385635    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fed73a15-8dd4-451b-8110-46a3f73c22c2-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.385873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385854    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.386037 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.385996    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.386100 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.386038    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.386235 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.386217    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.387319 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.387292    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:46:04.388297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.388266    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:04.388427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.388409    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "kube-api-access-7g4pc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:04.388599 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.388580    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token" pod "b9ecdaf9-8526-430b-9373-ec36c6897253" (UID: "b9ecdaf9-8526-430b-9373-ec36c6897253"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:04.486966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.486921    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.486966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.486960    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.486966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.486970    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.487194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.486978    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g4pc\" (UniqueName: \"kubernetes.io/projected/b9ecdaf9-8526-430b-9373-ec36c6897253-kube-api-access-7g4pc\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.487194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.486988    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.487194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.486997    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.487194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.487006    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.487194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.487014    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b9ecdaf9-8526-430b-9373-ec36c6897253-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:04.742516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.738861    5057 generic.go:356] "Generic (PLEG): container finished" podID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerID="e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437" exitCode=0
Nov 01 22:46:04.742516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.739001    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" event={"ID":"fed73a15-8dd4-451b-8110-46a3f73c22c2","Type":"ContainerDied","Data":"e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437"}
Nov 01 22:46:04.742516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.739057    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7" event={"ID":"fed73a15-8dd4-451b-8110-46a3f73c22c2","Type":"ContainerDied","Data":"d77693612dce29e7eb838c438e0eb47fdfd1e4340f7da60fd183336e155dffd6"}
Nov 01 22:46:04.742516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.739090    5057 scope.go:122] "RemoveContainer" containerID="e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437"
Nov 01 22:46:04.742516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.739251    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"
Nov 01 22:46:04.745027 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.744994    5057 generic.go:356] "Generic (PLEG): container finished" podID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerID="715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947" exitCode=0
Nov 01 22:46:04.745135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.745076    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"
Nov 01 22:46:04.745135 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.745075    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" event={"ID":"b9ecdaf9-8526-430b-9373-ec36c6897253","Type":"ContainerDied","Data":"715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947"}
Nov 01 22:46:04.745247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.745137    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt" event={"ID":"b9ecdaf9-8526-430b-9373-ec36c6897253","Type":"ContainerDied","Data":"ec5a7ed2a57f53c20cad397ab44e7bbb4ef7c2ba22161e539fd9f105b5a9401d"}
Nov 01 22:46:04.758648 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.758627    5057 scope.go:122] "RemoveContainer" containerID="e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437"
Nov 01 22:46:04.759850 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:04.759821    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437\": container with ID starting with e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437 not found: ID does not exist" containerID="e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437"
Nov 01 22:46:04.759950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.759857    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437"} err="failed to get container status \"e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437\": rpc error: code = NotFound desc = could not find container \"e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437\": container with ID starting with e1fba817a0d896a8f83aba00ba4895a7bfdf196e04177aae765cf738df933437 not found: ID does not exist"
Nov 01 22:46:04.759950 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.759881    5057 scope.go:122] "RemoveContainer" containerID="715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947"
Nov 01 22:46:04.770932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.770566    5057 scope.go:122] "RemoveContainer" containerID="715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947"
Nov 01 22:46:04.770932 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:04.770867    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947\": container with ID starting with 715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947 not found: ID does not exist" containerID="715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947"
Nov 01 22:46:04.770932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.770894    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947"} err="failed to get container status \"715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947\": rpc error: code = NotFound desc = could not find container \"715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947\": container with ID starting with 715712d3f38fd68054e5151993e684a552c8958a7a819b5ae0c2ebac3df80947 not found: ID does not exist"
Nov 01 22:46:04.782189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.782159    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"]
Nov 01 22:46:04.790935 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.790903    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-q9hlt"]
Nov 01 22:46:04.805269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.805199    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"]
Nov 01 22:46:04.810949 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.810921    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b42cc4-openshift-default-8c76c4757-8d4k7"]
Nov 01 22:46:04.875722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.875688    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 22:46:04.875882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:04.875739    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 22:46:05.296513 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:05.296469    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:05.399642 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:05.399609    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" path="/var/lib/kubelet/pods/b9ecdaf9-8526-430b-9373-ec36c6897253/volumes"
Nov 01 22:46:05.400264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:05.400243    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" path="/var/lib/kubelet/pods/fed73a15-8dd4-451b-8110-46a3f73c22c2/volumes"
Nov 01 22:46:05.875542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:05.875478    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused" start-of-body=
Nov 01 22:46:05.875707 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:05.875567    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.112:15021/healthz/ready\": dial tcp 10.132.0.112:15021: connect: connection refused"
Nov 01 22:46:06.252605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.249001    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:06.253846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.253821    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:46:06.383876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.383837    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.383876 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.383874    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.383935    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384097    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384179    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384226    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384325    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384357    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384374    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384407    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384458    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket\") pod \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\" (UID: \"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7\") "
Nov 01 22:46:06.384962 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.384560    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:06.385322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.385133    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.385322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.385159    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.385322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.385175    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.385575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.385479    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:06.385733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.385692    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:06.387050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.387027    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:06.387161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.387068    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:06.387161 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.387131    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:46:06.387276 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.387199    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw" pod "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" (UID: "2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7"). InnerVolumeSpecName "kube-api-access-4t9dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:06.486925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.486851    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.486925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.486935    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.487214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.486952    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.487214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.486968    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.487214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.486987    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.487214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.487002    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4t9dw\" (UniqueName: \"kubernetes.io/projected/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7-kube-api-access-4t9dw\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:06.766746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.766648    5057 generic.go:356] "Generic (PLEG): container finished" podID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerID="114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b" exitCode=0
Nov 01 22:46:06.766939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.766742    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" event={"ID":"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7","Type":"ContainerDied","Data":"114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b"}
Nov 01 22:46:06.766939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.766764    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"
Nov 01 22:46:06.766939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.766832    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt" event={"ID":"2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7","Type":"ContainerDied","Data":"11f404fd0ce23f45343fe6946ed7289068c5c6b52f0f487453e4d687f0f43b7e"}
Nov 01 22:46:06.766939 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.766879    5057 scope.go:122] "RemoveContainer" containerID="114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b"
Nov 01 22:46:06.779106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.778972    5057 scope.go:122] "RemoveContainer" containerID="6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"
Nov 01 22:46:06.793112 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.793085    5057 scope.go:122] "RemoveContainer" containerID="114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b"
Nov 01 22:46:06.793476 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:06.793455    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b\": container with ID starting with 114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b not found: ID does not exist" containerID="114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b"
Nov 01 22:46:06.793582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.793484    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b"} err="failed to get container status \"114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b\": rpc error: code = NotFound desc = could not find container \"114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b\": container with ID starting with 114bbaace4f31439a98d44b36daa915ba593700e57ed885e4a629c02d746232b not found: ID does not exist"
Nov 01 22:46:06.793582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.793519    5057 scope.go:122] "RemoveContainer" containerID="6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"
Nov 01 22:46:06.793833 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:06.793802    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199\": container with ID starting with 6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199 not found: ID does not exist" containerID="6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"
Nov 01 22:46:06.793906 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.793844    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199"} err="failed to get container status \"6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199\": rpc error: code = NotFound desc = could not find container \"6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199\": container with ID starting with 6736d78917525c8c5d0cbec6d9e88e29b349096f0f2ed6ad38a1c5525e57c199 not found: ID does not exist"
Nov 01 22:46:06.799821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.799791    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"]
Nov 01 22:46:06.806110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:06.806082    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a9c08e-openshift-default-7dbbf746c7-pnbxt"]
Nov 01 22:46:07.026118 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:07.026037    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:46:07.026544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:07.026521    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:46:07.027359 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:07.027338    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:46:07.039274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:07.039200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:07.208348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:07.208313    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:07.411321 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:07.410650    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" path="/var/lib/kubelet/pods/2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7/volumes"
Nov 01 22:46:08.006435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:08.004294    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:08.684639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:08.684602    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:09.641435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:09.635056    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:09.789401 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:09.789362    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:09.961596 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:09.961559    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:10.024517 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:10.022152    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:46:10.146221 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:10.146186    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:10.536243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:10.534727    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:10.706385 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:10.706353    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:11.651542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:11.651467    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:11.829848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:11.829813    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:12.482042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:12.482010    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:13.085205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.085155    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"]
Nov 01 22:46:13.088042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.087379    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" podUID="5eb7ec26-3c0c-414c-995c-503eeb590770" containerName="maas-api" containerID="cri-o://c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581" gracePeriod=30
Nov 01 22:46:13.186016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.184672    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:13.271517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.257661    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"]
Nov 01 22:46:13.271517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.258338    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc" podUID="70062ef4-2ebb-4b14-950a-fbadd78dbdc9" containerName="maas-api" containerID="cri-o://d839e50e1042d118dfdd89d85e05c19a275c3fe5d0078bd6b32b41176208343c" gracePeriod=30
Nov 01 22:46:13.359420 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.359119    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:13.515582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.515318    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:13.754043 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.753996    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:13.818199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.817793    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:46:13.857947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.857416    5057 generic.go:356] "Generic (PLEG): container finished" podID="70062ef4-2ebb-4b14-950a-fbadd78dbdc9" containerID="d839e50e1042d118dfdd89d85e05c19a275c3fe5d0078bd6b32b41176208343c" exitCode=0
Nov 01 22:46:13.857947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.857657    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc" event={"ID":"70062ef4-2ebb-4b14-950a-fbadd78dbdc9","Type":"ContainerDied","Data":"d839e50e1042d118dfdd89d85e05c19a275c3fe5d0078bd6b32b41176208343c"}
Nov 01 22:46:13.859858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.859788    5057 generic.go:356] "Generic (PLEG): container finished" podID="5eb7ec26-3c0c-414c-995c-503eeb590770" containerID="c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581" exitCode=0
Nov 01 22:46:13.859858 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.859846    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" event={"ID":"5eb7ec26-3c0c-414c-995c-503eeb590770","Type":"ContainerDied","Data":"c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581"}
Nov 01 22:46:13.860170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.859890    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9" event={"ID":"5eb7ec26-3c0c-414c-995c-503eeb590770","Type":"ContainerDied","Data":"c13767ed15f59e306219fd7caaf739ddb329e744a162ab23fde41b3101e85291"}
Nov 01 22:46:13.860170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.859922    5057 scope.go:122] "RemoveContainer" containerID="c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581"
Nov 01 22:46:13.860170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.860085    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"
Nov 01 22:46:13.885968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.883837    5057 scope.go:122] "RemoveContainer" containerID="c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581"
Nov 01 22:46:13.885968 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:13.884731    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581\": container with ID starting with c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581 not found: ID does not exist" containerID="c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581"
Nov 01 22:46:13.885968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.884774    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581"} err="failed to get container status \"c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581\": rpc error: code = NotFound desc = could not find container \"c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581\": container with ID starting with c6edb8be00c1eae9c781f9b5f79387e5f9ab9548472d5835529d4b1baeec5581 not found: ID does not exist"
Nov 01 22:46:13.960129 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.960092    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs\") pod \"5eb7ec26-3c0c-414c-995c-503eeb590770\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") "
Nov 01 22:46:13.960875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.960852    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca\") pod \"5eb7ec26-3c0c-414c-995c-503eeb590770\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") "
Nov 01 22:46:13.962629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.962504    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca" pod "5eb7ec26-3c0c-414c-995c-503eeb590770" (UID: "5eb7ec26-3c0c-414c-995c-503eeb590770"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:13.962904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.962879    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\") pod \"5eb7ec26-3c0c-414c-995c-503eeb590770\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") "
Nov 01 22:46:13.978204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.978177    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd\" (UniqueName: \"kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd\") pod \"5eb7ec26-3c0c-414c-995c-503eeb590770\" (UID: \"5eb7ec26-3c0c-414c-995c-503eeb590770\") "
Nov 01 22:46:13.979056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.976031    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs" pod "5eb7ec26-3c0c-414c-995c-503eeb590770" (UID: "5eb7ec26-3c0c-414c-995c-503eeb590770"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:13.979056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.979043    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5eb7ec26-3c0c-414c-995c-503eeb590770-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:13.979056 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.976027    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls" pod "5eb7ec26-3c0c-414c-995c-503eeb590770" (UID: "5eb7ec26-3c0c-414c-995c-503eeb590770"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:13.981793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.981759    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd" pod "5eb7ec26-3c0c-414c-995c-503eeb590770" (UID: "5eb7ec26-3c0c-414c-995c-503eeb590770"). InnerVolumeSpecName "kube-api-access-g9rfd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:13.988881 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.988859    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:46:13.996875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:13.996848    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:14.080352 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.080260    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd\" (UniqueName: \"kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd\") pod \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") "
Nov 01 22:46:14.080557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.080454    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca\") pod \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") "
Nov 01 22:46:14.080557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.080503    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls\") pod \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") "
Nov 01 22:46:14.080557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.080547    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs\") pod \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\" (UID: \"70062ef4-2ebb-4b14-950a-fbadd78dbdc9\") "
Nov 01 22:46:14.080909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.080883    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca" pod "70062ef4-2ebb-4b14-950a-fbadd78dbdc9" (UID: "70062ef4-2ebb-4b14-950a-fbadd78dbdc9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:14.081170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.081145    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.081291 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.081174    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.081291 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.081188    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g9rfd\" (UniqueName: \"kubernetes.io/projected/5eb7ec26-3c0c-414c-995c-503eeb590770-kube-api-access-g9rfd\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.081291 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.081202    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eb7ec26-3c0c-414c-995c-503eeb590770-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.082706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.082677    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls" pod "70062ef4-2ebb-4b14-950a-fbadd78dbdc9" (UID: "70062ef4-2ebb-4b14-950a-fbadd78dbdc9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:14.082706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.082702    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs" pod "70062ef4-2ebb-4b14-950a-fbadd78dbdc9" (UID: "70062ef4-2ebb-4b14-950a-fbadd78dbdc9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:14.082829 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.082762    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd" pod "70062ef4-2ebb-4b14-950a-fbadd78dbdc9" (UID: "70062ef4-2ebb-4b14-950a-fbadd78dbdc9"). InnerVolumeSpecName "kube-api-access-wf7kd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:14.183142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.182776    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.183142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.182821    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.183142 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.182840    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf7kd\" (UniqueName: \"kubernetes.io/projected/70062ef4-2ebb-4b14-950a-fbadd78dbdc9-kube-api-access-wf7kd\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:14.215413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.215373    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"]
Nov 01 22:46:14.245225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.244883    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8f2555-67cb8d8ddc-vcrf9"]
Nov 01 22:46:14.398413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.398327    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:14.872164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.872131    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"
Nov 01 22:46:14.872164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.872148    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc" event={"ID":"70062ef4-2ebb-4b14-950a-fbadd78dbdc9","Type":"ContainerDied","Data":"fa15ae8367a99f59975aa433f2a71e1dc40d7d534d58c4a7407fc2786f56a62b"}
Nov 01 22:46:14.872443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.872219    5057 scope.go:122] "RemoveContainer" containerID="d839e50e1042d118dfdd89d85e05c19a275c3fe5d0078bd6b32b41176208343c"
Nov 01 22:46:14.909929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.909896    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"]
Nov 01 22:46:14.915401 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:14.915371    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-fa8cda-764d4c5568-crznc"]
Nov 01 22:46:15.397542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:15.397485    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5eb7ec26-3c0c-414c-995c-503eeb590770" path="/var/lib/kubelet/pods/5eb7ec26-3c0c-414c-995c-503eeb590770/volumes"
Nov 01 22:46:15.398042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:15.398014    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="70062ef4-2ebb-4b14-950a-fbadd78dbdc9" path="/var/lib/kubelet/pods/70062ef4-2ebb-4b14-950a-fbadd78dbdc9/volumes"
Nov 01 22:46:15.448976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:15.448938    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:15.572752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:15.572714    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"]
Nov 01 22:46:15.582910 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:15.582875    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"]
Nov 01 22:46:15.750966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:15.750932    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:16.498545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:16.498409    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:17.337162 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:17.337124    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:18.085414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:18.085371    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:18.694096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:18.694045    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:19.532805 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:19.532763    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:20.331214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:20.331176    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:20.814245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:20.814207    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:46:21.473476 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:21.473442    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:21.703581 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:21.703542    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:21.923170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:21.923000    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:22.883441 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:22.883405    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:24.157250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:24.157141    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:24.461218 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:24.459725    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:25.167199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:25.167165    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:26.011376 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:26.011336    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:26.157853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:26.157811    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:26.302285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:26.302190    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:26.503877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:26.503847    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:26.652540 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:26.652400    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:26.853568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:26.853531    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:27.022050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:27.022017    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:27.258925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:27.258842    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:28.072663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:28.072628    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:29.017301 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:29.017265    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:29.791918 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:29.791881    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:30.568516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:30.562981    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:30.596197 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:30.596150    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" containerID="cri-o://6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6" gracePeriod=45
Nov 01 22:46:30.606395 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:30.606351    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main" containerID="cri-o://96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12" gracePeriod=45
Nov 01 22:46:30.680972 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:30.680932    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.56:8000/ready\": dial tcp 10.132.0.56:8000: connect: connection refused"
Nov 01 22:46:30.929773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:30.929360    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:46:31.013665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.013051    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:46:31.013665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.013300    5057 generic.go:356] "Generic (PLEG): container finished" podID="04f907c0-418a-4a90-8d9f-204203d366d4" containerID="6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6" exitCode=0
Nov 01 22:46:31.013665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.013399    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"
Nov 01 22:46:31.013665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.013437    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" event={"ID":"04f907c0-418a-4a90-8d9f-204203d366d4","Type":"ContainerDied","Data":"6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6"}
Nov 01 22:46:31.013665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.013470    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c" event={"ID":"04f907c0-418a-4a90-8d9f-204203d366d4","Type":"ContainerDied","Data":"5e03b1bb05da7a64fffea4d7bfe6ccd20ec0807c0ebcdf5cdc8761110eb5fdf7"}
Nov 01 22:46:31.013665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.013509    5057 scope.go:122] "RemoveContainer" containerID="6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6"
Nov 01 22:46:31.016256 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.016224    5057 generic.go:356] "Generic (PLEG): container finished" podID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerID="96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12" exitCode=0
Nov 01 22:46:31.016440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.016411    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" event={"ID":"dde04f9a-4461-49da-8aba-7bc3e19edb59","Type":"ContainerDied","Data":"96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12"}
Nov 01 22:46:31.016694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.016461    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9" event={"ID":"dde04f9a-4461-49da-8aba-7bc3e19edb59","Type":"ContainerDied","Data":"d237627a20ef51186ae423fa63c119e62d36de65df92624c0df5d0820a22bf0b"}
Nov 01 22:46:31.016694 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.016626    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"
Nov 01 22:46:31.044260 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.043917    5057 scope.go:122] "RemoveContainer" containerID="6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6"
Nov 01 22:46:31.045366 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:31.044988    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6\": container with ID starting with 6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6 not found: ID does not exist" containerID="6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6"
Nov 01 22:46:31.045366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.045030    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6"} err="failed to get container status \"6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6\": rpc error: code = NotFound desc = could not find container \"6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6\": container with ID starting with 6d0edda9f821719cd256265a0d1c25cb88119714438c22be808a0be0946afba6 not found: ID does not exist"
Nov 01 22:46:31.045366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.045056    5057 scope.go:122] "RemoveContainer" containerID="96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12"
Nov 01 22:46:31.059440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.059413    5057 scope.go:122] "RemoveContainer" containerID="96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12"
Nov 01 22:46:31.059834 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:31.059808    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12\": container with ID starting with 96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12 not found: ID does not exist" containerID="96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12"
Nov 01 22:46:31.059945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.059843    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12"} err="failed to get container status \"96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12\": rpc error: code = NotFound desc = could not find container \"96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12\": container with ID starting with 96a8533564330d42323f768600589c74146039760b2e07c1180602b89c5f6d12 not found: ID does not exist"
Nov 01 22:46:31.082605 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.082543    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home\") pod \"04f907c0-418a-4a90-8d9f-204203d366d4\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") "
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.083035    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4\" (UniqueName: \"kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4\") pod \"04f907c0-418a-4a90-8d9f-204203d366d4\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") "
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.083088    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir\") pod \"04f907c0-418a-4a90-8d9f-204203d366d4\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") "
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.083174    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache\") pod \"04f907c0-418a-4a90-8d9f-204203d366d4\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") "
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.083259    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs\" (UniqueName: \"kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs\") pod \"04f907c0-418a-4a90-8d9f-204203d366d4\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") "
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.083291    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm\") pod \"04f907c0-418a-4a90-8d9f-204203d366d4\" (UID: \"04f907c0-418a-4a90-8d9f-204203d366d4\") "
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.084017    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir" pod "04f907c0-418a-4a90-8d9f-204203d366d4" (UID: "04f907c0-418a-4a90-8d9f-204203d366d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.084530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.084197    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home" pod "04f907c0-418a-4a90-8d9f-204203d366d4" (UID: "04f907c0-418a-4a90-8d9f-204203d366d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.085063 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.084935    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache" pod "04f907c0-418a-4a90-8d9f-204203d366d4" (UID: "04f907c0-418a-4a90-8d9f-204203d366d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.085408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.085340    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.085408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.085370    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.085408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.085386    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.087664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.087598    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs" pod "04f907c0-418a-4a90-8d9f-204203d366d4" (UID: "04f907c0-418a-4a90-8d9f-204203d366d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:31.088878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.088852    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm" pod "04f907c0-418a-4a90-8d9f-204203d366d4" (UID: "04f907c0-418a-4a90-8d9f-204203d366d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.089036 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.089012    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4" pod "04f907c0-418a-4a90-8d9f-204203d366d4" (UID: "04f907c0-418a-4a90-8d9f-204203d366d4"). InnerVolumeSpecName "kube-api-access-r4lg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:31.185929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.185882    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir\") pod \"dde04f9a-4461-49da-8aba-7bc3e19edb59\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") "
Nov 01 22:46:31.186116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.185993    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home\") pod \"dde04f9a-4461-49da-8aba-7bc3e19edb59\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") "
Nov 01 22:46:31.186116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186067    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm\" (UniqueName: \"kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm\") pod \"dde04f9a-4461-49da-8aba-7bc3e19edb59\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") "
Nov 01 22:46:31.186116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186102    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache\") pod \"dde04f9a-4461-49da-8aba-7bc3e19edb59\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") "
Nov 01 22:46:31.186322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186157    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm\") pod \"dde04f9a-4461-49da-8aba-7bc3e19edb59\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") "
Nov 01 22:46:31.186322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186187    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs\" (UniqueName: \"kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs\") pod \"dde04f9a-4461-49da-8aba-7bc3e19edb59\" (UID: \"dde04f9a-4461-49da-8aba-7bc3e19edb59\") "
Nov 01 22:46:31.186322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186181    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir" pod "dde04f9a-4461-49da-8aba-7bc3e19edb59" (UID: "dde04f9a-4461-49da-8aba-7bc3e19edb59"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.186322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186296    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home" pod "dde04f9a-4461-49da-8aba-7bc3e19edb59" (UID: "dde04f9a-4461-49da-8aba-7bc3e19edb59"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.186584 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186346    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache" pod "dde04f9a-4461-49da-8aba-7bc3e19edb59" (UID: "dde04f9a-4461-49da-8aba-7bc3e19edb59"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.186819 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186795    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.186908 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186823    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.186908 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186887    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4lg4\" (UniqueName: \"kubernetes.io/projected/04f907c0-418a-4a90-8d9f-204203d366d4-kube-api-access-r4lg4\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.187024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186910    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/04f907c0-418a-4a90-8d9f-204203d366d4-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.187024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186924    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/04f907c0-418a-4a90-8d9f-204203d366d4-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.187024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.186938    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.188573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.188533    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm" pod "dde04f9a-4461-49da-8aba-7bc3e19edb59" (UID: "dde04f9a-4461-49da-8aba-7bc3e19edb59"). InnerVolumeSpecName "kube-api-access-k8flm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:31.188573 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.188557    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs" pod "dde04f9a-4461-49da-8aba-7bc3e19edb59" (UID: "dde04f9a-4461-49da-8aba-7bc3e19edb59"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:31.188746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.188604    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm" pod "dde04f9a-4461-49da-8aba-7bc3e19edb59" (UID: "dde04f9a-4461-49da-8aba-7bc3e19edb59"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:31.288203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.288155    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8flm\" (UniqueName: \"kubernetes.io/projected/dde04f9a-4461-49da-8aba-7bc3e19edb59-kube-api-access-k8flm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.288203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.288191    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dde04f9a-4461-49da-8aba-7bc3e19edb59-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.288203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.288201    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dde04f9a-4461-49da-8aba-7bc3e19edb59-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:31.361839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.361751    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"]
Nov 01 22:46:31.367990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.367955    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/simulator-w3-00e49d-kserve-56f8f7b454-llkv9"]
Nov 01 22:46:31.384686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.384651    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"]
Nov 01 22:46:31.399044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.399013    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" path="/var/lib/kubelet/pods/dde04f9a-4461-49da-8aba-7bc3e19edb59/volumes"
Nov 01 22:46:31.399351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.399336    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-00e49d/premium-w3-00e49d-kserve-7dd8b4d787-7lm8c"]
Nov 01 22:46:31.486269 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:31.486225    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:32.376521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:32.373339    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:32.533115 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:32.533077    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:33.403266 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:33.403227    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" path="/var/lib/kubelet/pods/04f907c0-418a-4a90-8d9f-204203d366d4/volumes"
Nov 01 22:46:33.571992 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:33.571957    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:34.386109 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:34.386068    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:34.588149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:34.588114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:34.946700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:34.946662    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:35.194433 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:35.194391    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:35.487688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:35.487650    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:35.770277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:35.770159    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:36.120005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:36.119911    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:36.710049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:36.710014    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:37.464031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:37.463995    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:37.978521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:37.975415    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:39.059550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:39.059513    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:39.815799 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:39.815763    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:40.020971 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:40.020938    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:46:40.870804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:40.870768    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"]
Nov 01 22:46:40.884624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:40.884594    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"]
Nov 01 22:46:40.900424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:40.900389    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"]
Nov 01 22:46:40.972641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:40.972532    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:41.126911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:41.126803    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:41.568417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:41.568284    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"]
Nov 01 22:46:41.569315 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:41.569271    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" podUID="c8b7ba43-4774-4977-a0db-1473f20b138b" containerName="maas-api" containerID="cri-o://dda2c9e9d273d83f7eb31da7761ebd7adcb5a8d74f48041b0a6dcb97f2b41250" gracePeriod=30
Nov 01 22:46:41.661292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:41.661243    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:41.965925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:41.965850    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:42.108253 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.108215    5057 generic.go:356] "Generic (PLEG): container finished" podID="c8b7ba43-4774-4977-a0db-1473f20b138b" containerID="dda2c9e9d273d83f7eb31da7761ebd7adcb5a8d74f48041b0a6dcb97f2b41250" exitCode=0
Nov 01 22:46:42.108408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.108364    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" event={"ID":"c8b7ba43-4774-4977-a0db-1473f20b138b","Type":"ContainerDied","Data":"dda2c9e9d273d83f7eb31da7761ebd7adcb5a8d74f48041b0a6dcb97f2b41250"}
Nov 01 22:46:42.108453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.108416    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w" event={"ID":"c8b7ba43-4774-4977-a0db-1473f20b138b","Type":"ContainerDied","Data":"620a1245619549020004386368ec89cc2fc0c9113e1fadce3604280d0d4327cb"}
Nov 01 22:46:42.108453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.108439    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="620a1245619549020004386368ec89cc2fc0c9113e1fadce3604280d0d4327cb"
Nov 01 22:46:42.117038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.117002    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:46:42.129349 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.129316    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:42.232273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.232187    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca\") pod \"c8b7ba43-4774-4977-a0db-1473f20b138b\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") "
Nov 01 22:46:42.232423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.232307    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs\") pod \"c8b7ba43-4774-4977-a0db-1473f20b138b\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") "
Nov 01 22:46:42.232423 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.232337    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\") pod \"c8b7ba43-4774-4977-a0db-1473f20b138b\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") "
Nov 01 22:46:42.232526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.232504    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb\" (UniqueName: \"kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb\") pod \"c8b7ba43-4774-4977-a0db-1473f20b138b\" (UID: \"c8b7ba43-4774-4977-a0db-1473f20b138b\") "
Nov 01 22:46:42.232582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.232539    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca" pod "c8b7ba43-4774-4977-a0db-1473f20b138b" (UID: "c8b7ba43-4774-4977-a0db-1473f20b138b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:42.233153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.233114    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c8b7ba43-4774-4977-a0db-1473f20b138b-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:42.234835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.234799    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls" pod "c8b7ba43-4774-4977-a0db-1473f20b138b" (UID: "c8b7ba43-4774-4977-a0db-1473f20b138b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:42.234958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.234908    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs" pod "c8b7ba43-4774-4977-a0db-1473f20b138b" (UID: "c8b7ba43-4774-4977-a0db-1473f20b138b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:42.234958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.234938    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb" pod "c8b7ba43-4774-4977-a0db-1473f20b138b" (UID: "c8b7ba43-4774-4977-a0db-1473f20b138b"). InnerVolumeSpecName "kube-api-access-6slhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:42.286893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.286848    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:42.334357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.334315    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6slhb\" (UniqueName: \"kubernetes.io/projected/c8b7ba43-4774-4977-a0db-1473f20b138b-kube-api-access-6slhb\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:42.334357 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.334357    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:42.334652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.334372    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c8b7ba43-4774-4977-a0db-1473f20b138b-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:42.544174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:42.544071    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:43.160516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.160386    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:43.186514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.186002    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"
Nov 01 22:46:43.272515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.270104    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"]
Nov 01 22:46:43.273542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.273514    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w6-c9a7ce-7697ccfb66-m5d5w"]
Nov 01 22:46:43.331627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.331591    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:43.398300 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.398264    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c8b7ba43-4774-4977-a0db-1473f20b138b" path="/var/lib/kubelet/pods/c8b7ba43-4774-4977-a0db-1473f20b138b/volumes"
Nov 01 22:46:43.680206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:43.680167    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:44.107198 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:44.107161    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:44.244398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:44.244364    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:45.078695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:45.078626    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:45.818698 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:45.818661    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:46.222519 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.211214    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:46.372851 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.372816    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"]
Nov 01 22:46:46.373061 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.373024    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" containerID="cri-o://20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3" gracePeriod=30
Nov 01 22:46:46.384861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.384826    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:46.535571 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.535472    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:46.660807 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.660775    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"]
Nov 01 22:46:46.661232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.661190    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy" containerID="cri-o://1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e" gracePeriod=30
Nov 01 22:46:46.698374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.698332    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:46.891169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:46.891086    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:47.730304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:47.730265    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:48.357769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:48.357719    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:49.050820 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:49.050784    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"]
Nov 01 22:46:49.271270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:49.271233    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:49.462922 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:49.462879    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:49.995878 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:49.995841    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:46:50.577071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:50.577023    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:50.811171 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:50.811133    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:51.006884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.006851    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:51.683629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.683563    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:46:51.803042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803009    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803328 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803302    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:51.803428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803308    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803456    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803517    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803557    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803682    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803810 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803790    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803843    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.803925 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.803880    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm\") pod \"0cbdcb39-f084-4626-9541-d5b0be671b8d\" (UID: \"0cbdcb39-f084-4626-9541-d5b0be671b8d\") "
Nov 01 22:46:51.804691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.804515    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.805244 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.805183    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:51.805440 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.805404    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:51.805877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.805657    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:51.806083 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.806042    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:51.807368 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.807326    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "kube-api-access-84dlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:51.807909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.807876    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:46:51.808965 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.808930    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:51.810331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.810210    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token" pod "0cbdcb39-f084-4626-9541-d5b0be671b8d" (UID: "0cbdcb39-f084-4626-9541-d5b0be671b8d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:51.906288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906244    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906277    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906286    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906298    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84dlm\" (UniqueName: \"kubernetes.io/projected/0cbdcb39-f084-4626-9541-d5b0be671b8d-kube-api-access-84dlm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906308    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906316    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cbdcb39-f084-4626-9541-d5b0be671b8d-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906325    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.906676 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.906334    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cbdcb39-f084-4626-9541-d5b0be671b8d-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:51.942109 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:51.942078    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:46:52.007274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007238    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007294    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007344    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007400    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007459    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007539    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007572    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007721    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.007840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007727    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:52.007840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.007806    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data\") pod \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\" (UID: \"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c\") "
Nov 01 22:46:52.008264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008231    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:46:52.008429 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008406    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:52.008730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008706    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.008899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008881    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.008998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008986    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.009110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008889    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:52.009210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.008905    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:52.011243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.011214    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "kube-api-access-25kxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:52.011511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.011473    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:52.011511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.011483    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:46:52.011685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.011509    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token" pod "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" (UID: "38efc380-1a2f-4b7d-b2c5-f2aa86810e6c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:52.110857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.110820    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.110857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.110854    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25kxf\" (UniqueName: \"kubernetes.io/projected/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-kube-api-access-25kxf\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.110857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.110865    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.111193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.110874    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.111193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.110884    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.111193 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.110907    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:52.344265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.344226    5057 generic.go:356] "Generic (PLEG): container finished" podID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerID="20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3" exitCode=0
Nov 01 22:46:52.344458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.344279    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"
Nov 01 22:46:52.344458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.344308    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" event={"ID":"0cbdcb39-f084-4626-9541-d5b0be671b8d","Type":"ContainerDied","Data":"20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3"}
Nov 01 22:46:52.344458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.344361    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" event={"ID":"0cbdcb39-f084-4626-9541-d5b0be671b8d","Type":"ContainerDied","Data":"bd9b317db6f217778f68d0205fb3a78977e556b981de37438b6652db153d7808"}
Nov 01 22:46:52.344458 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.344382    5057 scope.go:122] "RemoveContainer" containerID="20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3"
Nov 01 22:46:52.345776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.345749    5057 generic.go:356] "Generic (PLEG): container finished" podID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerID="1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e" exitCode=0
Nov 01 22:46:52.345927 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.345794    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"
Nov 01 22:46:52.345927 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.345831    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" event={"ID":"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c","Type":"ContainerDied","Data":"1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e"}
Nov 01 22:46:52.345927 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.345885    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p" event={"ID":"38efc380-1a2f-4b7d-b2c5-f2aa86810e6c","Type":"ContainerDied","Data":"9b894ca0bba9057c01d33c0a0647fba610a85dfa68eba2fea4f33894f7f10020"}
Nov 01 22:46:52.357769 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.357691    5057 scope.go:122] "RemoveContainer" containerID="20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3"
Nov 01 22:46:52.358083 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:52.358060    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3\": container with ID starting with 20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3 not found: ID does not exist" containerID="20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3"
Nov 01 22:46:52.358153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.358091    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3"} err="failed to get container status \"20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3\": rpc error: code = NotFound desc = could not find container \"20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3\": container with ID starting with 20ee103bd01b35655150b66750aaa29cc71438aa3d4fb027c8c12406ea10ffd3 not found: ID does not exist"
Nov 01 22:46:52.358153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.358110    5057 scope.go:122] "RemoveContainer" containerID="1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e"
Nov 01 22:46:52.368040 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.368014    5057 scope.go:122] "RemoveContainer" containerID="1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e"
Nov 01 22:46:52.368365 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:46:52.368338    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e\": container with ID starting with 1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e not found: ID does not exist" containerID="1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e"
Nov 01 22:46:52.368464 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.368371    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e"} err="failed to get container status \"1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e\": rpc error: code = NotFound desc = could not find container \"1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e\": container with ID starting with 1ddedaaa83ced2b12f029c6055b2f52a396df7ec97c99215ba9d513927ace62e not found: ID does not exist"
Nov 01 22:46:52.388783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.388751    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"]
Nov 01 22:46:52.396904 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.396876    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x"]
Nov 01 22:46:52.424322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.424287    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"]
Nov 01 22:46:52.436822 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.436789    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-tt47p"]
Nov 01 22:46:52.599155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.599054    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:46:52.599155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.599119    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-x488x" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:46:52.839233 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:52.839190    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:53.399706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:53.399670    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" path="/var/lib/kubelet/pods/0cbdcb39-f084-4626-9541-d5b0be671b8d/volumes"
Nov 01 22:46:53.400403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:53.400376    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" path="/var/lib/kubelet/pods/38efc380-1a2f-4b7d-b2c5-f2aa86810e6c/volumes"
Nov 01 22:46:53.463098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:53.462620    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:54.555189 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:54.555152    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:54.732719 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:54.732679    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:54.882598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:54.882475    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:55.676008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:55.675969    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:55.908515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:55.902744    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main" containerID="cri-o://6145270f009e847ea2b66a55889ebd7ad832cd16a4b517d0d3a0c6c888bc3a1e" gracePeriod=45
Nov 01 22:46:55.943121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:55.923687    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" containerID="cri-o://c81667e1c4e11bc62e632f257c89ffe7b34d93334643f95317fb3365d79975ad" gracePeriod=45
Nov 01 22:46:55.943121 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:55.933950    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" containerID="cri-o://5ecc203b041ff8dab8b023547762ffd0da631890a087922ec6ce095c1996e706" gracePeriod=45
Nov 01 22:46:55.983963 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:55.979171    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:56.070230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.069909    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": dial tcp 10.132.0.54:8000: connect: connection refused"
Nov 01 22:46:56.151011 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.148618    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": dial tcp 10.132.0.57:8000: connect: connection refused"
Nov 01 22:46:56.151011 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.150009    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:56.347615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.342029    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:56.387741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.383552    5057 generic.go:356] "Generic (PLEG): container finished" podID="d0b83430-f00d-4de0-a667-28278090f818" containerID="6145270f009e847ea2b66a55889ebd7ad832cd16a4b517d0d3a0c6c888bc3a1e" exitCode=0
Nov 01 22:46:56.387741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.383695    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" event={"ID":"d0b83430-f00d-4de0-a667-28278090f818","Type":"ContainerDied","Data":"6145270f009e847ea2b66a55889ebd7ad832cd16a4b517d0d3a0c6c888bc3a1e"}
Nov 01 22:46:56.387741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.385364    5057 generic.go:356] "Generic (PLEG): container finished" podID="f32326fc-3d31-4930-9150-da39082bc0d4" containerID="c81667e1c4e11bc62e632f257c89ffe7b34d93334643f95317fb3365d79975ad" exitCode=0
Nov 01 22:46:56.387741 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.385626    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" event={"ID":"f32326fc-3d31-4930-9150-da39082bc0d4","Type":"ContainerDied","Data":"c81667e1c4e11bc62e632f257c89ffe7b34d93334643f95317fb3365d79975ad"}
Nov 01 22:46:56.388073 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.387793    5057 generic.go:356] "Generic (PLEG): container finished" podID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerID="5ecc203b041ff8dab8b023547762ffd0da631890a087922ec6ce095c1996e706" exitCode=0
Nov 01 22:46:56.388073 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.387920    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" event={"ID":"5f3a24a7-886c-450f-b301-a9b83ec2f243","Type":"ContainerDied","Data":"5ecc203b041ff8dab8b023547762ffd0da631890a087922ec6ce095c1996e706"}
Nov 01 22:46:56.530777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.530746    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:46:56.537546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.536333    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:46:56.551371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.551337    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.657951    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache\") pod \"d0b83430-f00d-4de0-a667-28278090f818\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658013    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home\") pod \"5f3a24a7-886c-450f-b301-a9b83ec2f243\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658058    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs\") pod \"d0b83430-f00d-4de0-a667-28278090f818\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658105    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs\") pod \"5f3a24a7-886c-450f-b301-a9b83ec2f243\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658140    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq\" (UniqueName: \"kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq\") pod \"d0b83430-f00d-4de0-a667-28278090f818\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658190    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home\") pod \"f32326fc-3d31-4930-9150-da39082bc0d4\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658331    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache\") pod \"5f3a24a7-886c-450f-b301-a9b83ec2f243\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658388    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm\") pod \"d0b83430-f00d-4de0-a667-28278090f818\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658486    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf\" (UniqueName: \"kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf\") pod \"5f3a24a7-886c-450f-b301-a9b83ec2f243\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658550    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache\") pod \"f32326fc-3d31-4930-9150-da39082bc0d4\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658633    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm\") pod \"5f3a24a7-886c-450f-b301-a9b83ec2f243\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658672    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir\") pod \"5f3a24a7-886c-450f-b301-a9b83ec2f243\" (UID: \"5f3a24a7-886c-450f-b301-a9b83ec2f243\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658762    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home\") pod \"d0b83430-f00d-4de0-a667-28278090f818\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658806    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs\" (UniqueName: \"kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs\") pod \"f32326fc-3d31-4930-9150-da39082bc0d4\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658874    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir\") pod \"f32326fc-3d31-4930-9150-da39082bc0d4\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658932    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm\") pod \"f32326fc-3d31-4930-9150-da39082bc0d4\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.658967    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns\" (UniqueName: \"kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns\") pod \"f32326fc-3d31-4930-9150-da39082bc0d4\" (UID: \"f32326fc-3d31-4930-9150-da39082bc0d4\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.659037    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir\") pod \"d0b83430-f00d-4de0-a667-28278090f818\" (UID: \"d0b83430-f00d-4de0-a667-28278090f818\") "
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.660111    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir" pod "d0b83430-f00d-4de0-a667-28278090f818" (UID: "d0b83430-f00d-4de0-a667-28278090f818"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.660345    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache" pod "f32326fc-3d31-4930-9150-da39082bc0d4" (UID: "f32326fc-3d31-4930-9150-da39082bc0d4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.661877    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir" pod "5f3a24a7-886c-450f-b301-a9b83ec2f243" (UID: "5f3a24a7-886c-450f-b301-a9b83ec2f243"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.662157    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home" pod "5f3a24a7-886c-450f-b301-a9b83ec2f243" (UID: "5f3a24a7-886c-450f-b301-a9b83ec2f243"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.662409    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache" pod "d0b83430-f00d-4de0-a667-28278090f818" (UID: "d0b83430-f00d-4de0-a667-28278090f818"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.662618    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir" pod "f32326fc-3d31-4930-9150-da39082bc0d4" (UID: "f32326fc-3d31-4930-9150-da39082bc0d4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.662812    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home" pod "d0b83430-f00d-4de0-a667-28278090f818" (UID: "d0b83430-f00d-4de0-a667-28278090f818"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.662979    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home" pod "f32326fc-3d31-4930-9150-da39082bc0d4" (UID: "f32326fc-3d31-4930-9150-da39082bc0d4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.665209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.663448    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache" pod "5f3a24a7-886c-450f-b301-a9b83ec2f243" (UID: "5f3a24a7-886c-450f-b301-a9b83ec2f243"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.672631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.671406    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm" pod "d0b83430-f00d-4de0-a667-28278090f818" (UID: "d0b83430-f00d-4de0-a667-28278090f818"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.672786 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.672659    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns" pod "f32326fc-3d31-4930-9150-da39082bc0d4" (UID: "f32326fc-3d31-4930-9150-da39082bc0d4"). InnerVolumeSpecName "kube-api-access-gx5ns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:56.688840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.686702    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs" pod "d0b83430-f00d-4de0-a667-28278090f818" (UID: "d0b83430-f00d-4de0-a667-28278090f818"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:56.688840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.686835    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs" pod "5f3a24a7-886c-450f-b301-a9b83ec2f243" (UID: "5f3a24a7-886c-450f-b301-a9b83ec2f243"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:56.688840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.688659    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs" pod "f32326fc-3d31-4930-9150-da39082bc0d4" (UID: "f32326fc-3d31-4930-9150-da39082bc0d4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:46:56.688840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.688805    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq" pod "d0b83430-f00d-4de0-a667-28278090f818" (UID: "d0b83430-f00d-4de0-a667-28278090f818"). InnerVolumeSpecName "kube-api-access-zhbhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:56.690041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.689996    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm" pod "f32326fc-3d31-4930-9150-da39082bc0d4" (UID: "f32326fc-3d31-4930-9150-da39082bc0d4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.700379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.700348    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf" pod "5f3a24a7-886c-450f-b301-a9b83ec2f243" (UID: "5f3a24a7-886c-450f-b301-a9b83ec2f243"). InnerVolumeSpecName "kube-api-access-ttckf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:46:56.706373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.706215    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm" pod "5f3a24a7-886c-450f-b301-a9b83ec2f243" (UID: "5f3a24a7-886c-450f-b301-a9b83ec2f243"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:46:56.748718 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.748617    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:56.761523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761470    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761524    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761538    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f32326fc-3d31-4930-9150-da39082bc0d4-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761553    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761567    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761580    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx5ns\" (UniqueName: \"kubernetes.io/projected/f32326fc-3d31-4930-9150-da39082bc0d4-kube-api-access-gx5ns\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761596    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761610    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761622    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761636    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b83430-f00d-4de0-a667-28278090f818-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761687    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f3a24a7-886c-450f-b301-a9b83ec2f243-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761701    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhbhq\" (UniqueName: \"kubernetes.io/projected/d0b83430-f00d-4de0-a667-28278090f818-kube-api-access-zhbhq\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761713    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761746    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761759    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b83430-f00d-4de0-a667-28278090f818-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761773    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttckf\" (UniqueName: \"kubernetes.io/projected/5f3a24a7-886c-450f-b301-a9b83ec2f243-kube-api-access-ttckf\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.761781 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761785    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f32326fc-3d31-4930-9150-da39082bc0d4-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:56.762606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:56.761798    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f3a24a7-886c-450f-b301-a9b83ec2f243-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:46:57.408041 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.407947    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"
Nov 01 22:46:57.408933 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.408889    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm" event={"ID":"5f3a24a7-886c-450f-b301-a9b83ec2f243","Type":"ContainerDied","Data":"df6dc51e37d982a579413faa48e7fd00cde67d147536dc2e474ab3d7f26caa86"}
Nov 01 22:46:57.409089 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.408947    5057 scope.go:122] "RemoveContainer" containerID="5ecc203b041ff8dab8b023547762ffd0da631890a087922ec6ce095c1996e706"
Nov 01 22:46:57.411782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.411750    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7" event={"ID":"d0b83430-f00d-4de0-a667-28278090f818","Type":"ContainerDied","Data":"14efd7874407d0ad9aa339155458e50ae43b17af403e59c74d23c5147b2d353f"}
Nov 01 22:46:57.411942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.411807    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"
Nov 01 22:46:57.413729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.413701    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm" event={"ID":"f32326fc-3d31-4930-9150-da39082bc0d4","Type":"ContainerDied","Data":"d2a359c080ceed7e27af987742f1c302b8b0529bfa1608d86c75263c610aec44"}
Nov 01 22:46:57.413828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.413796    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"
Nov 01 22:46:57.432290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.432269    5057 scope.go:122] "RemoveContainer" containerID="6145270f009e847ea2b66a55889ebd7ad832cd16a4b517d0d3a0c6c888bc3a1e"
Nov 01 22:46:57.445852 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.445829    5057 scope.go:122] "RemoveContainer" containerID="c81667e1c4e11bc62e632f257c89ffe7b34d93334643f95317fb3365d79975ad"
Nov 01 22:46:57.462711 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.461999    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"]
Nov 01 22:46:57.465728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.465705    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/distinct-w2-3d8490-kserve-685c9d87d5-hrhn7"]
Nov 01 22:46:57.474578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.474552    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"]
Nov 01 22:46:57.480457 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.480429    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/simulator-w2-3d8490-kserve-85f75b9bc9-4h2jm"]
Nov 01 22:46:57.491153 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.491127    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"]
Nov 01 22:46:57.497657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.497627    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-3d8490/premium-w2-3d8490-kserve-5f7bc9dcd6-q48nm"]
Nov 01 22:46:57.648126 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.648092    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:57.942877 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:57.942840    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:58.131638 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:58.131598    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:58.265125 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:58.265077    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:58.415548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:58.414709    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"]
Nov 01 22:46:58.415548 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:58.415002    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy" containerID="cri-o://4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7" gracePeriod=30
Nov 01 22:46:58.480049 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:58.480010    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:58.634576 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:58.628016    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:59.288285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:59.288244    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:46:59.397359 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:59.397327    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" path="/var/lib/kubelet/pods/5f3a24a7-886c-450f-b301-a9b83ec2f243/volumes"
Nov 01 22:46:59.397839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:59.397816    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0b83430-f00d-4de0-a667-28278090f818" path="/var/lib/kubelet/pods/d0b83430-f00d-4de0-a667-28278090f818/volumes"
Nov 01 22:46:59.398237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:46:59.398217    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" path="/var/lib/kubelet/pods/f32326fc-3d31-4930-9150-da39082bc0d4/volumes"
Nov 01 22:47:00.079523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:00.079472    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:01.108151 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:01.108114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:01.452388 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:01.452255    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:01.603227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:01.603185    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:02.182824 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:02.182786    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:02.916932 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:02.916814    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:03.593241 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.585974    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:03.751259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.751228    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:47:03.775050 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775014    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.775246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775082    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.775246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775133    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.775246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775212    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.775411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775302    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.775411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775362    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.775535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775438    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:03.775535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775444    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.776752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775548    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.776752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.775608    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket\") pod \"7f545ced-4074-41f3-be68-590aec5756e2\" (UID: \"7f545ced-4074-41f3-be68-590aec5756e2\") "
Nov 01 22:47:03.776752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.776649    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:03.776752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.776666    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7f545ced-4074-41f3-be68-590aec5756e2-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.777342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.776857    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:03.777342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.776870    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:03.777998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.777963    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:03.779167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.779145    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "kube-api-access-nmjkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:03.779293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.779242    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:03.780462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.780430    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:03.780621 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.780548    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo" pod "7f545ced-4074-41f3-be68-590aec5756e2" (UID: "7f545ced-4074-41f3-be68-590aec5756e2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:47:03.885526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885423    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885460    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7f545ced-4074-41f3-be68-590aec5756e2-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885476    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885518    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885532    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885546    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7f545ced-4074-41f3-be68-590aec5756e2-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885560    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmjkb\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-kube-api-access-nmjkb\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:03.885958 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:03.885573    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7f545ced-4074-41f3-be68-590aec5756e2-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.074010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.073877    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main" containerID="cri-o://a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5" gracePeriod=45
Nov 01 22:47:04.347747 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.347718    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:47:04.396202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.396166    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f\" (UniqueName: \"kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f\") pod \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") "
Nov 01 22:47:04.397139 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.397112    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home\") pod \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") "
Nov 01 22:47:04.397748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.397711    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache\") pod \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") "
Nov 01 22:47:04.398272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.397762    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm\") pod \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") "
Nov 01 22:47:04.398272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.397855    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs\") pod \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") "
Nov 01 22:47:04.398272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.397884    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir\") pod \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\" (UID: \"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6\") "
Nov 01 22:47:04.400848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.397565    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home" pod "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" (UID: "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:04.400848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.398543    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache" pod "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" (UID: "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:04.406354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.406308    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir" pod "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" (UID: "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:04.407326 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.407298    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.407326 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.407328    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.407550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.407343    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.408326 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.408285    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f" pod "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" (UID: "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6"). InnerVolumeSpecName "kube-api-access-kks9f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:04.411374 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.411333    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs" pod "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" (UID: "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:04.412022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.411994    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm" pod "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" (UID: "b0e94f7e-58f3-407e-b3c1-7aac0082c4b6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:04.446405 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.446206    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:04.496453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.496412    5057 generic.go:356] "Generic (PLEG): container finished" podID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerID="a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5" exitCode=0
Nov 01 22:47:04.496662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.496612    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" event={"ID":"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6","Type":"ContainerDied","Data":"a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5"}
Nov 01 22:47:04.496662 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.496655    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q" event={"ID":"b0e94f7e-58f3-407e-b3c1-7aac0082c4b6","Type":"ContainerDied","Data":"38e1ac3ecaeb7bd9899d53a0e22c2f154e0ff2df17d814cab2a86700850f39c0"}
Nov 01 22:47:04.496746 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.496678    5057 scope.go:122] "RemoveContainer" containerID="a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5"
Nov 01 22:47:04.496832 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.496818    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"
Nov 01 22:47:04.504664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.504631    5057 generic.go:356] "Generic (PLEG): container finished" podID="7f545ced-4074-41f3-be68-590aec5756e2" containerID="4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7" exitCode=0
Nov 01 22:47:04.504820 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.504797    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" event={"ID":"7f545ced-4074-41f3-be68-590aec5756e2","Type":"ContainerDied","Data":"4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7"}
Nov 01 22:47:04.504890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.504851    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf" event={"ID":"7f545ced-4074-41f3-be68-590aec5756e2","Type":"ContainerDied","Data":"77c1ce074464dfac61f054adb1a09516a78022bb962caceed5da4aa9947f7a8c"}
Nov 01 22:47:04.504954 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.504942    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"
Nov 01 22:47:04.510532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.508745    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kks9f\" (UniqueName: \"kubernetes.io/projected/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-kube-api-access-kks9f\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.510532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.508772    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.510532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.508789    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:04.523019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.520065    5057 scope.go:122] "RemoveContainer" containerID="a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5"
Nov 01 22:47:04.523019 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:04.520675    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5\": container with ID starting with a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5 not found: ID does not exist" containerID="a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5"
Nov 01 22:47:04.523019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.520728    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5"} err="failed to get container status \"a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5\": rpc error: code = NotFound desc = could not find container \"a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5\": container with ID starting with a3fd3b308f09c77a31a4ebf63e167172d7a79a5605a65a0fa6ba56d549fdcbc5 not found: ID does not exist"
Nov 01 22:47:04.523019 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.520752    5057 scope.go:122] "RemoveContainer" containerID="4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7"
Nov 01 22:47:04.529853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.529826    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"]
Nov 01 22:47:04.536478 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.536449    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-381659/distinct-w1-381659-kserve-6bb4786f94-jxx4q"]
Nov 01 22:47:04.539080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.539061    5057 scope.go:122] "RemoveContainer" containerID="4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7"
Nov 01 22:47:04.539478 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:04.539435    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7\": container with ID starting with 4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7 not found: ID does not exist" containerID="4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7"
Nov 01 22:47:04.539604 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.539544    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7"} err="failed to get container status \"4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7\": rpc error: code = NotFound desc = could not find container \"4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7\": container with ID starting with 4196e5f791507b88fc0dcf37218b1ffca57e384a656fbe7a29f8984021425bf7 not found: ID does not exist"
Nov 01 22:47:04.547254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.547220    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"]
Nov 01 22:47:04.554051 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:04.554024    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-00e49d-openshift-default-5bd6688d47-wj7rf"]
Nov 01 22:47:05.148079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.148048    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"]
Nov 01 22:47:05.148794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.148758    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb" podUID="ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" containerName="maas-api" containerID="cri-o://127dd2b493855314e7968468d0f9534a9894f06702c3581f036c0c79b1b4bd19" gracePeriod=30
Nov 01 22:47:05.260948 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.260906    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:05.529597 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.518154    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:05.536988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.536953    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7f545ced-4074-41f3-be68-590aec5756e2" path="/var/lib/kubelet/pods/7f545ced-4074-41f3-be68-590aec5756e2/volumes"
Nov 01 22:47:05.537984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.537958    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" path="/var/lib/kubelet/pods/b0e94f7e-58f3-407e-b3c1-7aac0082c4b6/volumes"
Nov 01 22:47:05.561863 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.561829    5057 generic.go:356] "Generic (PLEG): container finished" podID="ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" containerID="127dd2b493855314e7968468d0f9534a9894f06702c3581f036c0c79b1b4bd19" exitCode=0
Nov 01 22:47:05.562173 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.562152    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb" event={"ID":"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd","Type":"ContainerDied","Data":"127dd2b493855314e7968468d0f9534a9894f06702c3581f036c0c79b1b4bd19"}
Nov 01 22:47:05.636062 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.636028    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:47:05.696159 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.696124    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls\") pod \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") "
Nov 01 22:47:05.696348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.696182    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca\") pod \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") "
Nov 01 22:47:05.696348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.696213    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4\" (UniqueName: \"kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4\") pod \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") "
Nov 01 22:47:05.696348 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.696270    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs\") pod \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\" (UID: \"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd\") "
Nov 01 22:47:05.696732 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.696693    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca" pod "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" (UID: "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:05.698691 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.698661    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls" pod "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" (UID: "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:05.699046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.699020    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4" pod "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" (UID: "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd"). InnerVolumeSpecName "kube-api-access-fd4d4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:05.699164 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.699076    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs" pod "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" (UID: "ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:05.721559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.721474    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:05.797716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.797632    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:05.797716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.797662    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:05.797716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.797673    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd4d4\" (UniqueName: \"kubernetes.io/projected/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-kube-api-access-fd4d4\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:05.797716 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.797684    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:05.872574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:05.866021    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:06.343071 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:06.343031    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:06.572897 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:06.572862    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"
Nov 01 22:47:06.573110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:06.572900    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb" event={"ID":"ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd","Type":"ContainerDied","Data":"1165b27bbf55023f2f55bd348984e48ead527f0b413159396d2a6ed2676e5e8f"}
Nov 01 22:47:06.573110 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:06.572977    5057 scope.go:122] "RemoveContainer" containerID="127dd2b493855314e7968468d0f9534a9894f06702c3581f036c0c79b1b4bd19"
Nov 01 22:47:06.608449 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:06.608401    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"]
Nov 01 22:47:06.611909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:06.611880    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w6-3fa400-c8d55d8f5-78ldb"]
Nov 01 22:47:07.129520 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:07.126869    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:07.397522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:07.397427    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" path="/var/lib/kubelet/pods/ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd/volumes"
Nov 01 22:47:07.983195 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:07.982627    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:08.763345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:08.763303    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:09.209093 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.209051    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"]
Nov 01 22:47:09.209629 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.209590    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd" podUID="a27ee460-17f6-4b56-ae1e-8886c70a5da1" containerName="maas-api" containerID="cri-o://6c7f3d3c316738e0923321b69bb4e4b2f135600baa95504b7b9e8b0ecf991c2c" gracePeriod=30
Nov 01 22:47:09.372544 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.372413    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:09.484569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.483254    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"]
Nov 01 22:47:09.484569 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.483841    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp" podUID="2947187c-8bf2-42c3-9ac4-6b045474baa2" containerName="maas-api" containerID="cri-o://eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7" gracePeriod=30
Nov 01 22:47:09.576909 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.576868    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:09.603890 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.603854    5057 generic.go:356] "Generic (PLEG): container finished" podID="a27ee460-17f6-4b56-ae1e-8886c70a5da1" containerID="6c7f3d3c316738e0923321b69bb4e4b2f135600baa95504b7b9e8b0ecf991c2c" exitCode=0
Nov 01 22:47:09.604065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.603966    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd" event={"ID":"a27ee460-17f6-4b56-ae1e-8886c70a5da1","Type":"ContainerDied","Data":"6c7f3d3c316738e0923321b69bb4e4b2f135600baa95504b7b9e8b0ecf991c2c"}
Nov 01 22:47:09.604065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.604005    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd" event={"ID":"a27ee460-17f6-4b56-ae1e-8886c70a5da1","Type":"ContainerDied","Data":"d3315de19d08e371951f9df7cc23cc16b2900b7601ac80f5cfe9cc1b9d9db248"}
Nov 01 22:47:09.604065 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.604021    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3315de19d08e371951f9df7cc23cc16b2900b7601ac80f5cfe9cc1b9d9db248"
Nov 01 22:47:09.625156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.624844    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:47:09.715717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.714800    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:09.761013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.759743    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs\") pod \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") "
Nov 01 22:47:09.761013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.759832    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls\") pod \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") "
Nov 01 22:47:09.761013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.759886    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca\") pod \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") "
Nov 01 22:47:09.761013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.760032    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc\" (UniqueName: \"kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc\") pod \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\" (UID: \"a27ee460-17f6-4b56-ae1e-8886c70a5da1\") "
Nov 01 22:47:09.767403 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.767333    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca" pod "a27ee460-17f6-4b56-ae1e-8886c70a5da1" (UID: "a27ee460-17f6-4b56-ae1e-8886c70a5da1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:09.864888 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.864834    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a27ee460-17f6-4b56-ae1e-8886c70a5da1-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:09.880216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.880154    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs" pod "a27ee460-17f6-4b56-ae1e-8886c70a5da1" (UID: "a27ee460-17f6-4b56-ae1e-8886c70a5da1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:09.880602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.880559    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc" pod "a27ee460-17f6-4b56-ae1e-8886c70a5da1" (UID: "a27ee460-17f6-4b56-ae1e-8886c70a5da1"). InnerVolumeSpecName "kube-api-access-6v4lc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:09.880855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.880819    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls" pod "a27ee460-17f6-4b56-ae1e-8886c70a5da1" (UID: "a27ee460-17f6-4b56-ae1e-8886c70a5da1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:09.971557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.966792    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:09.971557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.966831    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v4lc\" (UniqueName: \"kubernetes.io/projected/a27ee460-17f6-4b56-ae1e-8886c70a5da1-kube-api-access-6v4lc\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:09.971557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.966847    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a27ee460-17f6-4b56-ae1e-8886c70a5da1-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:09.976232 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:09.975829    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:47:10.056261 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:10.056138    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:47:10.068631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.068588    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls\") pod \"2947187c-8bf2-42c3-9ac4-6b045474baa2\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") "
Nov 01 22:47:10.069415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.069388    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca\") pod \"2947187c-8bf2-42c3-9ac4-6b045474baa2\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") "
Nov 01 22:47:10.070292 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.070166    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca" pod "2947187c-8bf2-42c3-9ac4-6b045474baa2" (UID: "2947187c-8bf2-42c3-9ac4-6b045474baa2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:10.070470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.070450    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs\") pod \"2947187c-8bf2-42c3-9ac4-6b045474baa2\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") "
Nov 01 22:47:10.071005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.070986    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs\" (UniqueName: \"kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs\") pod \"2947187c-8bf2-42c3-9ac4-6b045474baa2\" (UID: \"2947187c-8bf2-42c3-9ac4-6b045474baa2\") "
Nov 01 22:47:10.072842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.072723    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls" pod "2947187c-8bf2-42c3-9ac4-6b045474baa2" (UID: "2947187c-8bf2-42c3-9ac4-6b045474baa2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:10.073091 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.073053    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs" pod "2947187c-8bf2-42c3-9ac4-6b045474baa2" (UID: "2947187c-8bf2-42c3-9ac4-6b045474baa2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:10.073874 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.073852    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:10.074035 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.074018    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2947187c-8bf2-42c3-9ac4-6b045474baa2-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:10.074143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.074130    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2947187c-8bf2-42c3-9ac4-6b045474baa2-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:10.074532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.074476    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs" pod "2947187c-8bf2-42c3-9ac4-6b045474baa2" (UID: "2947187c-8bf2-42c3-9ac4-6b045474baa2"). InnerVolumeSpecName "kube-api-access-ldmgs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:10.121320 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.121276    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:10.175665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.175633    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldmgs\" (UniqueName: \"kubernetes.io/projected/2947187c-8bf2-42c3-9ac4-6b045474baa2-kube-api-access-ldmgs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:10.307062 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.306978    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:10.612510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.612409    5057 generic.go:356] "Generic (PLEG): container finished" podID="2947187c-8bf2-42c3-9ac4-6b045474baa2" containerID="eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7" exitCode=0
Nov 01 22:47:10.612510 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.612452    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"
Nov 01 22:47:10.612735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.612534    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp" event={"ID":"2947187c-8bf2-42c3-9ac4-6b045474baa2","Type":"ContainerDied","Data":"eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7"}
Nov 01 22:47:10.612735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.612588    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp" event={"ID":"2947187c-8bf2-42c3-9ac4-6b045474baa2","Type":"ContainerDied","Data":"38a890e6ee2a0832ef89b60f5a10eaa580f07c79e07f989fc19a2017678742ed"}
Nov 01 22:47:10.612735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.612614    5057 scope.go:122] "RemoveContainer" containerID="eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7"
Nov 01 22:47:10.612899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.612766    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"
Nov 01 22:47:10.632002 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.631977    5057 scope.go:122] "RemoveContainer" containerID="eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7"
Nov 01 22:47:10.632320 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:10.632295    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7\": container with ID starting with eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7 not found: ID does not exist" containerID="eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7"
Nov 01 22:47:10.632406 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.632345    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7"} err="failed to get container status \"eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7\": rpc error: code = NotFound desc = could not find container \"eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7\": container with ID starting with eac3be8abd2de9b45a0ddd51a886c8132504f8d0c25edd139fab69c745afd9b7 not found: ID does not exist"
Nov 01 22:47:10.657000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.656963    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"]
Nov 01 22:47:10.662601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.662568    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-381659-66b885f674-qmgxd"]
Nov 01 22:47:10.680455 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.680423    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"]
Nov 01 22:47:10.692222 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.692185    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-3d8490-7dd67486d9-hjsrp"]
Nov 01 22:47:10.728104 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:10.728063    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:11.147503 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:11.147449    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:11.396923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:11.396893    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2947187c-8bf2-42c3-9ac4-6b045474baa2" path="/var/lib/kubelet/pods/2947187c-8bf2-42c3-9ac4-6b045474baa2/volumes"
Nov 01 22:47:11.397324 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:11.397310    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a27ee460-17f6-4b56-ae1e-8886c70a5da1" path="/var/lib/kubelet/pods/a27ee460-17f6-4b56-ae1e-8886c70a5da1/volumes"
Nov 01 22:47:12.251029 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:12.250988    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:13.009351 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:13.009306    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:13.567602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:13.567554    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:13.728652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:13.728613    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:13.886648 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:13.886558    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:14.051717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.051659    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:14.266324 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.266286    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"]
Nov 01 22:47:14.266566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.266542    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy" containerID="cri-o://488fed8c6cfa120951584bdb26f9608f3e51e9851c7ec525e389ee46f3a792b1" gracePeriod=30
Nov 01 22:47:14.348990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.348957    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:14.712989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.712952    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"]
Nov 01 22:47:14.713475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713462    5057 cpu_manager.go:414] "containerMap: removing container" podUID="5eb7ec26-3c0c-414c-995c-503eeb590770" containerName="maas-api"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713476    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5eb7ec26-3c0c-414c-995c-503eeb590770" containerName="maas-api"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713524    5057 cpu_manager.go:414] "containerMap: removing container" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713530    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713538    5057 cpu_manager.go:414] "containerMap: removing container" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713544    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713553    5057 cpu_manager.go:414] "containerMap: removing container" podUID="c8b7ba43-4774-4977-a0db-1473f20b138b" containerName="maas-api"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713558    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c8b7ba43-4774-4977-a0db-1473f20b138b" containerName="maas-api"
Nov 01 22:47:14.713568 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713567    5057 cpu_manager.go:414] "containerMap: removing container" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713573    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713579    5057 cpu_manager.go:414] "containerMap: removing container" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713584    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713594    5057 cpu_manager.go:414] "containerMap: removing container" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713598    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713604    5057 cpu_manager.go:414] "containerMap: removing container" podUID="ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713608    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713618    5057 cpu_manager.go:414] "containerMap: removing container" podUID="2947187c-8bf2-42c3-9ac4-6b045474baa2" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713622    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2947187c-8bf2-42c3-9ac4-6b045474baa2" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713633    5057 cpu_manager.go:414] "containerMap: removing container" podUID="70062ef4-2ebb-4b14-950a-fbadd78dbdc9" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713637    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="70062ef4-2ebb-4b14-950a-fbadd78dbdc9" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713645    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a27ee460-17f6-4b56-ae1e-8886c70a5da1" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713649    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a27ee460-17f6-4b56-ae1e-8886c70a5da1" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713659    5057 cpu_manager.go:414] "containerMap: removing container" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713663    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713670    5057 cpu_manager.go:414] "containerMap: removing container" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713674    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713681    5057 cpu_manager.go:414] "containerMap: removing container" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713685    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713691    5057 cpu_manager.go:414] "containerMap: removing container" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713696    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713703    5057 cpu_manager.go:414] "containerMap: removing container" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713708    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713717    5057 cpu_manager.go:414] "containerMap: removing container" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713721    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713727    5057 cpu_manager.go:414] "containerMap: removing container" podUID="2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713731    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" containerName="maas-api"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713740    5057 cpu_manager.go:414] "containerMap: removing container" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713744    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713752    5057 cpu_manager.go:414] "containerMap: removing container" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713756    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713762    5057 cpu_manager.go:414] "containerMap: removing container" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main"
Nov 01 22:47:14.713815 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713767    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713838    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="b9ecdaf9-8526-430b-9373-ec36c6897253" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713846    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0e94f7e-58f3-407e-b3c1-7aac0082c4b6" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713853    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="c8b7ba43-4774-4977-a0db-1473f20b138b" containerName="maas-api"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713858    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0b83430-f00d-4de0-a667-28278090f818" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713862    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="7f545ced-4074-41f3-be68-590aec5756e2" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713869    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713874    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="ad6eb752-b2b7-4d80-8ced-a98d09c0c1bd" containerName="maas-api"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713881    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="2947187c-8bf2-42c3-9ac4-6b045474baa2" containerName="maas-api"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713888    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="70062ef4-2ebb-4b14-950a-fbadd78dbdc9" containerName="maas-api"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713893    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a27ee460-17f6-4b56-ae1e-8886c70a5da1" containerName="maas-api"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713900    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="86664679-f7d5-471e-b41b-98cdb6f48fdd" containerName="cleanup"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713905    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="04f907c0-418a-4a90-8d9f-204203d366d4" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713910    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="f32326fc-3d31-4930-9150-da39082bc0d4" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713915    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="0cbdcb39-f084-4626-9541-d5b0be671b8d" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713919    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="fed73a15-8dd4-451b-8110-46a3f73c22c2" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713924    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="38efc380-1a2f-4b7d-b2c5-f2aa86810e6c" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713928    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="2cdf9e8a-c61a-45cf-ac7c-ec5b4b84135e" containerName="maas-api"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713933    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="2fd7b3a6-7c49-48fa-81d3-9c18e3f3bfe7" containerName="istio-proxy"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713938    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="dde04f9a-4461-49da-8aba-7bc3e19edb59" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713943    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="5f3a24a7-886c-450f-b301-a9b83ec2f243" containerName="main"
Nov 01 22:47:14.714788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.713949    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="5eb7ec26-3c0c-414c-995c-503eeb590770" containerName="maas-api"
Nov 01 22:47:14.717361 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.717344    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.753943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.753909    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"]
Nov 01 22:47:14.769995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.769965    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:14.842283 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842238    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842324    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842360    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842405    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842436    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4lm6\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842480    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842554    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842649    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.842875 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.842703    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943536    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943597    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943625    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943651    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943673    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4lm6\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943706    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.943766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943743    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.944114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943809    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.944114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.943846    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.944114 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.944033    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.944473 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.944443    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.944663 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.944638    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.945149 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.945125    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.945818 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.945788    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.946942 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.946923    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.947593 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.947567    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.963318 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.963259    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.966272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.966245    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4lm6\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6\") pod \"e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") " pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:14.981204 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:14.981171    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:15.032203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.032169    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:15.061482 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.061442    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"]
Nov 01 22:47:15.067531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.067504    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.075700 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.075673    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9717e2-openshift-default-dockercfg-nbxqz\""
Nov 01 22:47:15.079931 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.079901    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"]
Nov 01 22:47:15.258445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258322    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.258445 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258373    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258512    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64qk5\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258603    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258633    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258677    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258708    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258821    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.259001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.258942    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360233    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360286    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360341    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64qk5\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360416    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360442    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360504    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360539    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360600    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.360786 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.360646    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.361095 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.361063    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.361237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.361207    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.361378 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.361243    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.361466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.361444    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.361702 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.361682    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.363259 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.363237    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.363362 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.363321    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.385546 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.385514    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64qk5\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.385792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.385772    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token\") pod \"e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") " pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.392123 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.392097    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:15.423150 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.423117    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"]
Nov 01 22:47:15.425267 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:47:15.425235    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b4a3bd2_7201_44e5_9f11_0a0bb1b38cf1.slice/crio-7fe942d97a5b0ffd8ae33b334c1d84e891b099cb3c64af652cc66bb1c368da0d WatchSource:0}: Error finding container 7fe942d97a5b0ffd8ae33b334c1d84e891b099cb3c64af652cc66bb1c368da0d: Status 404 returned error can't find the container with id 7fe942d97a5b0ffd8ae33b334c1d84e891b099cb3c64af652cc66bb1c368da0d
Nov 01 22:47:15.428080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.428045    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:47:15.428169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.428117    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:47:15.428169 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.428146    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:47:15.682047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.681962    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" event={"ID":"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1","Type":"ContainerStarted","Data":"3f1692904035236bb09599878660435a74d123611e3f6301bb80b77995635a2b"}
Nov 01 22:47:15.682199 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.682066    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" event={"ID":"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1","Type":"ContainerStarted","Data":"7fe942d97a5b0ffd8ae33b334c1d84e891b099cb3c64af652cc66bb1c368da0d"}
Nov 01 22:47:15.704717 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.704692    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"]
Nov 01 22:47:15.706833 ip-10-0-141-148 kubenswrapper[5057]: W1101 22:47:15.706801    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d8dc09_b7c3_434a_9595_d50e5cb06f77.slice/crio-68fe5c38d6fe9eeb526d3d954f7ce581f7763cf9ae3a47ad689ed63d28a8be82 WatchSource:0}: Error finding container 68fe5c38d6fe9eeb526d3d954f7ce581f7763cf9ae3a47ad689ed63d28a8be82: Status 404 returned error can't find the container with id 68fe5c38d6fe9eeb526d3d954f7ce581f7763cf9ae3a47ad689ed63d28a8be82
Nov 01 22:47:15.709628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.709593    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:47:15.709744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.709671    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:47:15.709744 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.709717    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 22:47:15.726597 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:15.719660    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podStartSLOduration=1.719644245 podStartE2EDuration="1.719644245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:47:15.711709666 +0000 UTC m=+1492.453405285" watchObservedRunningTime="2026-11-01 22:47:15.719644245 +0000 UTC m=+1492.461339865"
Nov 01 22:47:16.043515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.035672    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:16.043515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.035833    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:16.043515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.035891    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:16.210325 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.210295    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:16.690267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.690227    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" event={"ID":"a1d8dc09-b7c3-434a-9595-d50e5cb06f77","Type":"ContainerStarted","Data":"88c4294d47a87e6eeee3959a0fd5a846972f040672747435dbcd477393c0fe8b"}
Nov 01 22:47:16.690471 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.690286    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" event={"ID":"a1d8dc09-b7c3-434a-9595-d50e5cb06f77","Type":"ContainerStarted","Data":"68fe5c38d6fe9eeb526d3d954f7ce581f7763cf9ae3a47ad689ed63d28a8be82"}
Nov 01 22:47:16.750515 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:16.748700    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podStartSLOduration=1.748678774 podStartE2EDuration="1.748678774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 22:47:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 22:47:16.745222586 +0000 UTC m=+1493.486918205" watchObservedRunningTime="2026-11-01 22:47:16.748678774 +0000 UTC m=+1493.490374396"
Nov 01 22:47:17.034016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:17.033972    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:17.034340 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:17.034299    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:17.396017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:17.395932    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:17.396206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:17.395990    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:17.399917 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:17.399889    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:17.470333 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:17.470277    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:18.037919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:18.037885    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:18.038530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:18.038481    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:18.392671 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:18.392579    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:18.392919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:18.392893    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:19.035245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:19.034767    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:19.035245 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:19.034828    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:19.411234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:19.408239    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:19.411234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:19.408320    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:19.873309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:19.860379    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:20.057657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.020190    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"]
Nov 01 22:47:20.114559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.090008    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:20.114559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.090084    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:20.143082 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.133778    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:20.162516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.156914    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 22:47:20.162516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.156977    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:47:20.163590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.163554    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" probeResult="failure" output="Get \"https://10.132.0.60:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:47:20.212467 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.212431    5057 generic.go:356] "Generic (PLEG): container finished" podID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerID="488fed8c6cfa120951584bdb26f9608f3e51e9851c7ec525e389ee46f3a792b1" exitCode=0
Nov 01 22:47:20.221765 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.221727    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" event={"ID":"a18890f8-c3a4-4b2b-a261-983211e39f49","Type":"ContainerDied","Data":"488fed8c6cfa120951584bdb26f9608f3e51e9851c7ec525e389ee46f3a792b1"}
Nov 01 22:47:20.222028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.222011    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"]
Nov 01 22:47:20.234760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.234724    5057 scope.go:122] "RemoveContainer" containerID="4b579cb8778e742901c7037755e24861587b702e993695d5f3ba712ffd612695"
Nov 01 22:47:20.325060 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.323292    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"]
Nov 01 22:47:20.399695 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.399651    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" containerID="cri-o://bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516" gracePeriod=30
Nov 01 22:47:20.461606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.430398    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"]
Nov 01 22:47:20.490956 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.490911    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy" containerID="cri-o://b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1" gracePeriod=30
Nov 01 22:47:20.491302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.433061    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:20.498547 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.498516    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:20.498759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.498732    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:20.770536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.661467    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" probeResult="failure" output="Get \"https://10.132.0.58:8000/ready\": context deadline exceeded"
Nov 01 22:47:20.823557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.822691    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:20.999223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:20.981818    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Nov 01 22:47:21.031567 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.031462    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:21.048009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.047386    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:21.048009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.047447    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:21.147038 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.146688    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:47:21.284047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.283956    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.287927 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.287898    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.288212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.288192    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.288326 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.288313    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.288431 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.288418    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.288557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.288543    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.288674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.288662    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.305177 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.305148    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.305446 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.305414    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo\") pod \"a18890f8-c3a4-4b2b-a261-983211e39f49\" (UID: \"a18890f8-c3a4-4b2b-a261-983211e39f49\") "
Nov 01 22:47:21.345290 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.323898    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:21.347839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.328634    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:21.348053 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.336878    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:21.348179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.339643    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:21.348270 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.339689    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:21.348364 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.340576    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:21.348459 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.344147    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg" event={"ID":"a18890f8-c3a4-4b2b-a261-983211e39f49","Type":"ContainerDied","Data":"741da8eb4d2a30b713432df735987a7ddec84c87cb639f98aa3d320d2ad68d06"}
Nov 01 22:47:21.348627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.348598    5057 scope.go:122] "RemoveContainer" containerID="488fed8c6cfa120951584bdb26f9608f3e51e9851c7ec525e389ee46f3a792b1"
Nov 01 22:47:21.348842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.344224    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"
Nov 01 22:47:21.370539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.369249    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a18890f8-c3a4-4b2b-a261-983211e39f49-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.370539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.369287    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.370539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.369305    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.370539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.369320    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.370539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.369334    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.370539 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.369360    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"]
Nov 01 22:47:21.374530 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.372905    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" containerID="cri-o://239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426" gracePeriod=30
Nov 01 22:47:21.414688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.414655    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:21.414997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.414969    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:21.439572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.438924    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:21.439572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.439016    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:47:21.439572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.439072    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:21.485553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.483887    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv" pod "a18890f8-c3a4-4b2b-a261-983211e39f49" (UID: "a18890f8-c3a4-4b2b-a261-983211e39f49"). InnerVolumeSpecName "kube-api-access-gkfxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:21.485553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.484320    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"]
Nov 01 22:47:21.485553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.484628    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" containerID="cri-o://6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a" gracePeriod=30
Nov 01 22:47:21.489254 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.489229    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.489428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.489411    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.489579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.489562    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a18890f8-c3a4-4b2b-a261-983211e39f49-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.489696 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.489680    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkfxv\" (UniqueName: \"kubernetes.io/projected/a18890f8-c3a4-4b2b-a261-983211e39f49-kube-api-access-gkfxv\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:21.706806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.706769    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"]
Nov 01 22:47:21.709854 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.709818    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w6-3fa400-openshift-default-74796cdbc7-zcggg"]
Nov 01 22:47:21.972537 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:21.971456    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:22.041517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.040614    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:22.041517 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.040679    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:22.392824 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.392746    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:22.392824 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.392797    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:22.588119 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.588078    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:47:22.588641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.588136    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:47:22.987848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:22.980965    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:23.041868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.034983    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body=
Nov 01 22:47:23.041868 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.035045    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused"
Nov 01 22:47:23.144832 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.144758    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:23.393211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.393116    5057 patch_prober.go:29] interesting pod/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused" start-of-body=
Nov 01 22:47:23.393211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.393178    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.114:15021/healthz/ready\": dial tcp 10.132.0.114:15021: connect: connection refused"
Nov 01 22:47:23.406097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.405955    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" path="/var/lib/kubelet/pods/a18890f8-c3a4-4b2b-a261-983211e39f49/volumes"
Nov 01 22:47:23.692328 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.692245    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:47:23.699635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.699605    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:47:23.889436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.889406    5057 scope.go:122] "RemoveContainer" containerID="dda2c9e9d273d83f7eb31da7761ebd7adcb5a8d74f48041b0a6dcb97f2b41250"
Nov 01 22:47:23.900842 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:23.900818    5057 scope.go:122] "RemoveContainer" containerID="6c7f3d3c316738e0923321b69bb4e4b2f135600baa95504b7b9e8b0ecf991c2c"
Nov 01 22:47:24.038997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.038960    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:24.040192 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.040117    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:24.041042 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.041016    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:47:24.304661 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.304609    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:24.397424 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.397397    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:24.397730 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.397709    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:24.398442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.398422    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:47:24.671416 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.671315    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:24.800959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.800890    5057 patch_prober.go:29] interesting pod/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:47:24.800959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:24.800954    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:47:25.082731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:25.082696    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:25.314642 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:25.314610    5057 patch_prober.go:29] interesting pod/e2e-worker-w1-381659-openshift-default-894db6755-db6mw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:47:25.314804 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:25.314665    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:47:25.345191 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:25.345089    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:26.014749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.003114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:26.054441 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.054408    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:47:26.172718 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.172687    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:47:26.176722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176691    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.176899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176736    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.176899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176779    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.176899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176819    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.176899 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176863    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.177106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176905    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.177106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176994    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.177106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.176999    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.177106 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.177069    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.177672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.177191    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket\") pod \"c82a6301-b1dc-4116-9f89-95dc2ccec308\" (UID: \"c82a6301-b1dc-4116-9f89-95dc2ccec308\") "
Nov 01 22:47:26.177768 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.177747    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.178132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.177862    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.178132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.177976    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.178132 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.178088    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.178373 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.178260    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:26.180079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.180054    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.180170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.180104    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:47:26.180311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.180291    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "kube-api-access-hmfst". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.181179 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.181155    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy" pod "c82a6301-b1dc-4116-9f89-95dc2ccec308" (UID: "c82a6301-b1dc-4116-9f89-95dc2ccec308"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.278668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278630    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.278862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278688    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.278862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278734    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.278862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278770    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.278862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278800    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.278862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278839    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.279147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278916    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.279147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.278962    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.279147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279099    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket\") pod \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\" (UID: \"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84\") "
Nov 01 22:47:26.279147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279119    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.279435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279392    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.279545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279432    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:26.279683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279661    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.279753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279676    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.279753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279733    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279753    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279768    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279782    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c82a6301-b1dc-4116-9f89-95dc2ccec308-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279796    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279811    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmfst\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-kube-api-access-hmfst\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279841    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279850    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.279859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279859    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c82a6301-b1dc-4116-9f89-95dc2ccec308-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.280117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279867    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.280117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.279876    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c82a6301-b1dc-4116-9f89-95dc2ccec308-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.281577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.281553    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "kube-api-access-jmxcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.281660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.281569    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.281660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.281594    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:47:26.281660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.281573    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token" pod "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" (UID: "a9cb7b00-e3a7-4264-a7c3-34b53ca35d84"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.381952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.381860    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.381952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.381904    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.381952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.381919    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.381952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.381932    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.381952 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.381945    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.382246 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.381957    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmxcm\" (UniqueName: \"kubernetes.io/projected/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84-kube-api-access-jmxcm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.417023 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.416988    5057 generic.go:356] "Generic (PLEG): container finished" podID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerID="b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1" exitCode=0
Nov 01 22:47:26.417225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.417038    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"
Nov 01 22:47:26.417225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.417061    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" event={"ID":"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84","Type":"ContainerDied","Data":"b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1"}
Nov 01 22:47:26.417225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.417110    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt" event={"ID":"a9cb7b00-e3a7-4264-a7c3-34b53ca35d84","Type":"ContainerDied","Data":"ae02e60f104644f13fc17c4d5534d659f32d984c7c89432b238b55bdb65a99f2"}
Nov 01 22:47:26.417225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.417131    5057 scope.go:122] "RemoveContainer" containerID="b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1"
Nov 01 22:47:26.418750 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.418722    5057 generic.go:356] "Generic (PLEG): container finished" podID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerID="bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516" exitCode=0
Nov 01 22:47:26.418997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.418823    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" event={"ID":"c82a6301-b1dc-4116-9f89-95dc2ccec308","Type":"ContainerDied","Data":"bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516"}
Nov 01 22:47:26.418997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.418910    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw" event={"ID":"c82a6301-b1dc-4116-9f89-95dc2ccec308","Type":"ContainerDied","Data":"929cb46a19e1850746480366464b2b36622921fe8a04cd07b1fde8e367d69915"}
Nov 01 22:47:26.419122 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.419036    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"
Nov 01 22:47:26.429710 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.429685    5057 scope.go:122] "RemoveContainer" containerID="b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1"
Nov 01 22:47:26.430114 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:26.430084    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1\": container with ID starting with b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1 not found: ID does not exist" containerID="b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1"
Nov 01 22:47:26.430224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.430150    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1"} err="failed to get container status \"b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1\": rpc error: code = NotFound desc = could not find container \"b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1\": container with ID starting with b57c8acf7217d3b06fb5790c0d6d083fd29ada265a8fd21a95bf49aa8aaf0db1 not found: ID does not exist"
Nov 01 22:47:26.430224 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.430209    5057 scope.go:122] "RemoveContainer" containerID="bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516"
Nov 01 22:47:26.445988 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.445935    5057 scope.go:122] "RemoveContainer" containerID="bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516"
Nov 01 22:47:26.446439 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:26.446410    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516\": container with ID starting with bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516 not found: ID does not exist" containerID="bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516"
Nov 01 22:47:26.446969 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.446446    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516"} err="failed to get container status \"bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516\": rpc error: code = NotFound desc = could not find container \"bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516\": container with ID starting with bdd8892143ff3a7131efb705071bd4e2983df69afa853ff6dba9dc14ab211516 not found: ID does not exist"
Nov 01 22:47:26.492699 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.492660    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"]
Nov 01 22:47:26.512924 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.512893    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-db6mw"]
Nov 01 22:47:26.539911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.539879    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"]
Nov 01 22:47:26.555776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.555616    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-381659-openshift-default-894db6755-9nkgt"]
Nov 01 22:47:26.954552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.939826    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:47:26.954552 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.942378    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:47:26.989098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989064    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.989299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989121    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.989299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989158    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.989299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989185    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.989299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989212    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.989299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989258    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.989299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989289    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989393    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989453    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989487    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989486    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989566    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989596    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989586    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989627    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989659    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989678    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989720    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989756    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo\") pod \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\" (UID: \"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989832    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989859    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989895    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989923    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket\") pod \"c1b24ec1-6153-4575-83b2-4f894618dd72\" (UID: \"c1b24ec1-6153-4575-83b2-4f894618dd72\") "
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.989941    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990093    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990449    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990471    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1b24ec1-6153-4575-83b2-4f894618dd72-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990519    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990534    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990537    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990545    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990548    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990608    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.990690 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990628    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:26.992310 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.990732    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.992545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.992412    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.992545 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.992440    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "kube-api-access-46hxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.992677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.992596    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "kube-api-access-tqqg8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.992828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.992807    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:47:26.993265 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.993240    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:26.993382 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.993286    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.993800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.993773    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy" pod "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" (UID: "a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:26.994238 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:26.994195    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo" pod "c1b24ec1-6153-4575-83b2-4f894618dd72" (UID: "c1b24ec1-6153-4575-83b2-4f894618dd72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:47:27.076404 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.076218    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:27.092337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092301    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092349    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqqg8\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-kube-api-access-tqqg8\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092365    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46hxm\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-kube-api-access-46hxm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092377    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092389    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092400    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092414    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1b24ec1-6153-4575-83b2-4f894618dd72-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092426    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092447    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092460    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.092541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.092472    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1b24ec1-6153-4575-83b2-4f894618dd72-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:27.237452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.237412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:27.397978 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.397891    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" path="/var/lib/kubelet/pods/a9cb7b00-e3a7-4264-a7c3-34b53ca35d84/volumes"
Nov 01 22:47:27.398411 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.398395    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" path="/var/lib/kubelet/pods/c82a6301-b1dc-4116-9f89-95dc2ccec308/volumes"
Nov 01 22:47:27.428731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.428696    5057 generic.go:356] "Generic (PLEG): container finished" podID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerID="239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426" exitCode=0
Nov 01 22:47:27.428903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.428756    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"
Nov 01 22:47:27.428903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.428788    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" event={"ID":"c1b24ec1-6153-4575-83b2-4f894618dd72","Type":"ContainerDied","Data":"239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426"}
Nov 01 22:47:27.428903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.428857    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt" event={"ID":"c1b24ec1-6153-4575-83b2-4f894618dd72","Type":"ContainerDied","Data":"5b8c92fb9872fdd0bced41be6ebd26805653f5b07b28bdc9c64d6e7b6fecf0f1"}
Nov 01 22:47:27.428903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.428890    5057 scope.go:122] "RemoveContainer" containerID="239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426"
Nov 01 22:47:27.430432 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.430410    5057 generic.go:356] "Generic (PLEG): container finished" podID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerID="6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a" exitCode=0
Nov 01 22:47:27.430595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.430573    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"
Nov 01 22:47:27.430673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.430623    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" event={"ID":"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3","Type":"ContainerDied","Data":"6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a"}
Nov 01 22:47:27.430673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.430668    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl" event={"ID":"a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3","Type":"ContainerDied","Data":"73f540e53b4193f180762861ae2de3cb2d69ea835ce50e96029698d962877971"}
Nov 01 22:47:27.440748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.440664    5057 scope.go:122] "RemoveContainer" containerID="239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426"
Nov 01 22:47:27.441041 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:27.441011    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426\": container with ID starting with 239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426 not found: ID does not exist" containerID="239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426"
Nov 01 22:47:27.441125 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.441054    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426"} err="failed to get container status \"239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426\": rpc error: code = NotFound desc = could not find container \"239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426\": container with ID starting with 239248ae081cdc98c749440e2a895dc574eef19511f916303257a8995429f426 not found: ID does not exist"
Nov 01 22:47:27.441125 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.441077    5057 scope.go:122] "RemoveContainer" containerID="6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a"
Nov 01 22:47:27.452790 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.452766    5057 scope.go:122] "RemoveContainer" containerID="6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a"
Nov 01 22:47:27.453168 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:27.453145    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a\": container with ID starting with 6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a not found: ID does not exist" containerID="6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a"
Nov 01 22:47:27.453247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.453176    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a"} err="failed to get container status \"6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a\": rpc error: code = NotFound desc = could not find container \"6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a\": container with ID starting with 6154ddaeda9e192851ff3dbd4d8d7b66b95f3b51ee2e80b9a19ea0415f532e6a not found: ID does not exist"
Nov 01 22:47:27.453759 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.453737    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"]
Nov 01 22:47:27.463683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.463656    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-xkfzl"]
Nov 01 22:47:27.476004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.475976    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"]
Nov 01 22:47:27.482354 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.482325    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-3d8490-openshift-default-8474f778b5-h9rbt"]
Nov 01 22:47:27.986118 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:27.986077    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:29.079470 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:29.079434    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:29.289615 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:29.289565    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:29.398170 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:29.398092    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" path="/var/lib/kubelet/pods/a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3/volumes"
Nov 01 22:47:29.398618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:29.398604    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" path="/var/lib/kubelet/pods/c1b24ec1-6153-4575-83b2-4f894618dd72/volumes"
Nov 01 22:47:29.990001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:29.989966    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:30.897293 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:30.897254    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:31.867923 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:31.867888    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:32.624025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:32.623984    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:32.788392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:32.788356    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:32.952593 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:32.948931    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:33.355704 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:33.355615    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:33.887080 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:33.887036    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:34.619944 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:34.619907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:35.294862 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:35.294826    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:35.632262 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:35.632168    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main" containerID="cri-o://e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9" gracePeriod=45
Nov 01 22:47:35.916595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:35.916567    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:47:36.010657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010611    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir\") pod \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") "
Nov 01 22:47:36.010846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010715    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs\") pod \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") "
Nov 01 22:47:36.010846 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010776    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home\") pod \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") "
Nov 01 22:47:36.011001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010913    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r\" (UniqueName: \"kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r\") pod \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") "
Nov 01 22:47:36.011001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010960    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache\") pod \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") "
Nov 01 22:47:36.011001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010967    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir" pod "b5c05fa8-2874-4cb3-ba0a-f570a02800a6" (UID: "b5c05fa8-2874-4cb3-ba0a-f570a02800a6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.011001 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.010993    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm\") pod \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\" (UID: \"b5c05fa8-2874-4cb3-ba0a-f570a02800a6\") "
Nov 01 22:47:36.011205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.011173    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home" pod "b5c05fa8-2874-4cb3-ba0a-f570a02800a6" (UID: "b5c05fa8-2874-4cb3-ba0a-f570a02800a6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.011276 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.011202    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache" pod "b5c05fa8-2874-4cb3-ba0a-f570a02800a6" (UID: "b5c05fa8-2874-4cb3-ba0a-f570a02800a6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.011508 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.011472    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.011582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.011513    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.011582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.011531    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.013410 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.013384    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm" pod "b5c05fa8-2874-4cb3-ba0a-f570a02800a6" (UID: "b5c05fa8-2874-4cb3-ba0a-f570a02800a6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.013684 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.013653    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r" pod "b5c05fa8-2874-4cb3-ba0a-f570a02800a6" (UID: "b5c05fa8-2874-4cb3-ba0a-f570a02800a6"). InnerVolumeSpecName "kube-api-access-5gz5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:36.013782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.013699    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs" pod "b5c05fa8-2874-4cb3-ba0a-f570a02800a6" (UID: "b5c05fa8-2874-4cb3-ba0a-f570a02800a6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:36.058686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.058649    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main" containerID="cri-o://97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13" gracePeriod=45
Nov 01 22:47:36.112757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.112723    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gz5r\" (UniqueName: \"kubernetes.io/projected/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-kube-api-access-5gz5r\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.112757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.112754    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.112757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.112764    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b5c05fa8-2874-4cb3-ba0a-f570a02800a6-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.155033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.154930    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:36.305322 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.305294    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:47:36.377845 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.377809    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:36.416145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416046    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm\") pod \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") "
Nov 01 22:47:36.416295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416164    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs\" (UniqueName: \"kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs\") pod \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") "
Nov 01 22:47:36.416295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416195    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache\") pod \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") "
Nov 01 22:47:36.416295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416283    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2\" (UniqueName: \"kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2\") pod \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") "
Nov 01 22:47:36.416436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416374    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir\") pod \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") "
Nov 01 22:47:36.416436 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416429    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home\") pod \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\" (UID: \"61bbc377-4c9a-46b8-b8d7-05bbe87be152\") "
Nov 01 22:47:36.416559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416477    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache" pod "61bbc377-4c9a-46b8-b8d7-05bbe87be152" (UID: "61bbc377-4c9a-46b8-b8d7-05bbe87be152"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.416701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416675    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir" pod "61bbc377-4c9a-46b8-b8d7-05bbe87be152" (UID: "61bbc377-4c9a-46b8-b8d7-05bbe87be152"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.416823 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416799    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home" pod "61bbc377-4c9a-46b8-b8d7-05bbe87be152" (UID: "61bbc377-4c9a-46b8-b8d7-05bbe87be152"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.416967 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416949    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.417008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416972    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.417008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.416986    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.418312 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.418289    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm" pod "61bbc377-4c9a-46b8-b8d7-05bbe87be152" (UID: "61bbc377-4c9a-46b8-b8d7-05bbe87be152"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:47:36.418853 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.418829    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs" pod "61bbc377-4c9a-46b8-b8d7-05bbe87be152" (UID: "61bbc377-4c9a-46b8-b8d7-05bbe87be152"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:47:36.418959 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.418870    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2" pod "61bbc377-4c9a-46b8-b8d7-05bbe87be152" (UID: "61bbc377-4c9a-46b8-b8d7-05bbe87be152"). InnerVolumeSpecName "kube-api-access-zs6l2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:47:36.504870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.504836    5057 generic.go:356] "Generic (PLEG): container finished" podID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerID="e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9" exitCode=0
Nov 01 22:47:36.505079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.504881    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"
Nov 01 22:47:36.505079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.504931    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" event={"ID":"b5c05fa8-2874-4cb3-ba0a-f570a02800a6","Type":"ContainerDied","Data":"e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9"}
Nov 01 22:47:36.505079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.504972    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4" event={"ID":"b5c05fa8-2874-4cb3-ba0a-f570a02800a6","Type":"ContainerDied","Data":"e085e65fd2a08dc87ce42109796fde76069850274eab252d8ced399b3696d6ee"}
Nov 01 22:47:36.505079 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.504993    5057 scope.go:122] "RemoveContainer" containerID="e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9"
Nov 01 22:47:36.506536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.506474    5057 generic.go:356] "Generic (PLEG): container finished" podID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerID="97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13" exitCode=0
Nov 01 22:47:36.506673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.506547    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" event={"ID":"61bbc377-4c9a-46b8-b8d7-05bbe87be152","Type":"ContainerDied","Data":"97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13"}
Nov 01 22:47:36.506673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.506552    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"
Nov 01 22:47:36.506673 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.506578    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s" event={"ID":"61bbc377-4c9a-46b8-b8d7-05bbe87be152","Type":"ContainerDied","Data":"3dd5ca4e7814453d1f9438e58377b7866edec17358337fbb04ce5b5955655be7"}
Nov 01 22:47:36.517202 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.517098    5057 scope.go:122] "RemoveContainer" containerID="e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9"
Nov 01 22:47:36.517549 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:36.517519    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9\": container with ID starting with e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9 not found: ID does not exist" containerID="e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9"
Nov 01 22:47:36.517646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.517561    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9"} err="failed to get container status \"e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9\": rpc error: code = NotFound desc = could not find container \"e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9\": container with ID starting with e9f6139bc2eec995307797a9025e63277e8075a80779df95060d3e0deb24cfc9 not found: ID does not exist"
Nov 01 22:47:36.517646 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.517586    5057 scope.go:122] "RemoveContainer" containerID="97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13"
Nov 01 22:47:36.518004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.517971    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs6l2\" (UniqueName: \"kubernetes.io/projected/61bbc377-4c9a-46b8-b8d7-05bbe87be152-kube-api-access-zs6l2\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.518004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.517998    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61bbc377-4c9a-46b8-b8d7-05bbe87be152-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.518117 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.518016    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61bbc377-4c9a-46b8-b8d7-05bbe87be152-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:47:36.528817 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.528798    5057 scope.go:122] "RemoveContainer" containerID="97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13"
Nov 01 22:47:36.529179 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:36.529155    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13\": container with ID starting with 97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13 not found: ID does not exist" containerID="97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13"
Nov 01 22:47:36.529239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.529187    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13"} err="failed to get container status \"97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13\": rpc error: code = NotFound desc = could not find container \"97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13\": container with ID starting with 97850e68dd36804692bd2e9e13fefb38dc80ee8ae425aa8043871a601d9e8e13 not found: ID does not exist"
Nov 01 22:47:36.529356 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.529336    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"]
Nov 01 22:47:36.534379 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.534344    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/simulator-w6-b28a40-kserve-5fbcb5fdd-snxb4"]
Nov 01 22:47:36.545642 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.545612    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"]
Nov 01 22:47:36.552024 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:36.551994    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-b28a40/premium-w6-b28a40-kserve-74544b57bf-pjn4s"]
Nov 01 22:47:37.008627 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:37.008591    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:37.397908 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:37.397828    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" path="/var/lib/kubelet/pods/61bbc377-4c9a-46b8-b8d7-05bbe87be152/volumes"
Nov 01 22:47:37.398306 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:37.398291    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" path="/var/lib/kubelet/pods/b5c05fa8-2874-4cb3-ba0a-f570a02800a6/volumes"
Nov 01 22:47:37.679557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:37.679453    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:38.545366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:38.545329    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:39.258435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:39.258398    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:39.405426 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:39.405396    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:47:40.015352 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:47:40.015313    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:47:40.210028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:40.209985    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:41.101914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:41.101881    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:42.256218 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:42.256178    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:43.022709 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:43.022677    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:43.891824 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:43.891786    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:44.220669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:44.220574    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:44.380833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:44.380796    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:44.754302 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:44.754267    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:45.427782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:45.427741    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:46.094554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:46.094483    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:46.961834 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:46.961778    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:47.224109 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:47.224006    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:48.098557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:48.098520    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:48.635736 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:48.635699    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:48.864512 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:48.859193    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:49.004383 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:49.004338    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:49.129745 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:49.129698    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:49.500557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:49.499193    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:50.159278 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:50.159228    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:50.978005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:50.977958    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:51.766088 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:51.766049    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:52.475184 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:52.475143    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:53.213353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:53.213316    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:53.425749 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:53.425708    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:53.995111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:53.995065    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:54.782239 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:54.782204    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:55.533475 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:55.533429    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:55.991611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:55.991570    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:47:56.442203 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:56.442059    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:56.840857 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:56.840824    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:56.971727 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:56.971688    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:57.134257 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:57.134147    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"]
Nov 01 22:47:57.135104 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:57.135066    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy" containerID="cri-o://17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2" gracePeriod=30
Nov 01 22:47:57.141566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:57.140538    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"]
Nov 01 22:47:57.141566 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:57.140862    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" containerID="cri-o://cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd" gracePeriod=30
Nov 01 22:47:57.207408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:57.207375    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:57.524523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:57.524455    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:58.282264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:58.282226    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:59.400127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:59.400065    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:47:59.672014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:59.671926    5057 patch_prober.go:29] interesting pod/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:47:59.672014 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:47:59.671991    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:48:00.089916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:00.089874    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:00.774205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:00.774138    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:01.472443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:01.472405    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:02.193917 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.193881    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:02.439177 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.439139    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:48:02.481004 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.480949    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:48:02.550016 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.549971    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.550234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550045    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.550234 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550112    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.550372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550143    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.550372 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550335    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.550483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550309    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.550598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550418    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.550598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550486    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.550807 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550782    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.550905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550874    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.550986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.550966    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551248 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551221    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.551353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551012    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551284    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551353 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551347    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.551575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551377    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551410    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.551575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551453    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551575 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551518    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551602    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy\") pod \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\" (UID: \"c22e4429-a16f-49c7-b7e0-4b2eefc359ad\") "
Nov 01 22:48:02.551796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551633    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.551796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551669    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert\") pod \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\" (UID: \"eeaab9d5-3ace-4461-82a1-07e5b0437dac\") "
Nov 01 22:48:02.552215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.552188    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.552295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.552219    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.552295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.552235    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.552295 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.552249    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.559460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551631    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.559708    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.551916    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.552152    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.552589    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.559767    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.558305    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "kube-api-access-k8jrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.559215    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.559429    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.559801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.559468    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy" pod "c22e4429-a16f-49c7-b7e0-4b2eefc359ad" (UID: "c22e4429-a16f-49c7-b7e0-4b2eefc359ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.560297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.560178    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:48:02.560297 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.560265    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:48:02.560667 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.560633    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "kube-api-access-cb9rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:48:02.561624 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.561592    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy" pod "eeaab9d5-3ace-4461-82a1-07e5b0437dac" (UID: "eeaab9d5-3ace-4461-82a1-07e5b0437dac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:48:02.653143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653097    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8jrm\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-kube-api-access-k8jrm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653131    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653142    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cb9rh\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-kube-api-access-cb9rh\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653143 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653152    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653161    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653169    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653178    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653186    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653195    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653203    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653211    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653219    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eeaab9d5-3ace-4461-82a1-07e5b0437dac-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653229    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c22e4429-a16f-49c7-b7e0-4b2eefc359ad-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.653442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.653237    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eeaab9d5-3ace-4461-82a1-07e5b0437dac-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:48:02.679363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.679272    5057 generic.go:356] "Generic (PLEG): container finished" podID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerID="17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2" exitCode=0
Nov 01 22:48:02.679363 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.679325    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"
Nov 01 22:48:02.679612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.679417    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" event={"ID":"c22e4429-a16f-49c7-b7e0-4b2eefc359ad","Type":"ContainerDied","Data":"17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2"}
Nov 01 22:48:02.679612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.679485    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m" event={"ID":"c22e4429-a16f-49c7-b7e0-4b2eefc359ad","Type":"ContainerDied","Data":"ae94f6fb35bbff8c91f895b447271a4368eb195f66c017edad5e4fcdb1586bd8"}
Nov 01 22:48:02.679612 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.679560    5057 scope.go:122] "RemoveContainer" containerID="17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2"
Nov 01 22:48:02.681018 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.680886    5057 generic.go:356] "Generic (PLEG): container finished" podID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerID="cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd" exitCode=0
Nov 01 22:48:02.681018 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.680943    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"
Nov 01 22:48:02.681018 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.680964    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" event={"ID":"eeaab9d5-3ace-4461-82a1-07e5b0437dac","Type":"ContainerDied","Data":"cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd"}
Nov 01 22:48:02.681181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.681018    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r" event={"ID":"eeaab9d5-3ace-4461-82a1-07e5b0437dac","Type":"ContainerDied","Data":"3b8615106b0984ad2699cafc76c8f3314d5d191e00a0aaa2d3209043fa0c38e9"}
Nov 01 22:48:02.691816 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.691612    5057 scope.go:122] "RemoveContainer" containerID="17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2"
Nov 01 22:48:02.691939 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:48:02.691919    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2\": container with ID starting with 17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2 not found: ID does not exist" containerID="17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2"
Nov 01 22:48:02.691989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.691950    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2"} err="failed to get container status \"17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2\": rpc error: code = NotFound desc = could not find container \"17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2\": container with ID starting with 17486b6fecbe0cbdfcd6f5b17d322206b712ef4d9a270008da95672118c7eec2 not found: ID does not exist"
Nov 01 22:48:02.691989 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.691967    5057 scope.go:122] "RemoveContainer" containerID="cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd"
Nov 01 22:48:02.702392 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.702370    5057 scope.go:122] "RemoveContainer" containerID="cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd"
Nov 01 22:48:02.702741 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:48:02.702722    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd\": container with ID starting with cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd not found: ID does not exist" containerID="cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd"
Nov 01 22:48:02.702788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.702751    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd"} err="failed to get container status \"cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd\": rpc error: code = NotFound desc = could not find container \"cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd\": container with ID starting with cafbb777e317d4d0603b570b1ae5c4775eb70658e2262443cf722abd27600bdd not found: ID does not exist"
Nov 01 22:48:02.713046 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.713019    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"]
Nov 01 22:48:02.720206 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.720178    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-j4v8m"]
Nov 01 22:48:02.731826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.731787    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"]
Nov 01 22:48:02.742013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:02.741987    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-b28a40-openshift-default-668c9c7b7-mkq6r"]
Nov 01 22:48:03.111668 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:03.111637    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:03.397231 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:03.397148    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" path="/var/lib/kubelet/pods/c22e4429-a16f-49c7-b7e0-4b2eefc359ad/volumes"
Nov 01 22:48:03.397693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:03.397675    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" path="/var/lib/kubelet/pods/eeaab9d5-3ace-4461-82a1-07e5b0437dac/volumes"
Nov 01 22:48:03.787238 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:03.787193    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:04.576550 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:04.576483    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:05.303973 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:05.303932    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:05.984873 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:05.984823    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:06.689559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:06.684685    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:07.388578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:07.386699    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:08.317643 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:08.317607    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:08.988999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:08.988939    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:09.674802 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:09.674764    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:10.015919 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:48:10.015883    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:48:10.495054 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:10.495016    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:10.685848 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:10.685809    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:11.414675 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:11.414636    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:12.050250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:12.050213    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:12.694620 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:12.694584    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:13.380422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:13.380385    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:13.629733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:13.629695    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:14.180670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:14.180633    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:14.798104 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:14.797525    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:15.467828 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:15.467791    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:16.228685 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:16.228651    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:48:16.233461 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:16.233427    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:16.952595 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:16.952548    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:18.015999 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:18.015958    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:18.691133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:18.691064    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:19.495342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:19.495292    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:20.204304 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:20.204266    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:20.905291 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:20.905247    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:21.754281 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:21.754244    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:22.683427 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:22.683389    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:23.335535 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:23.335479    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:24.048945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:24.048905    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:24.906943 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:24.906878    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:25.679586 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:25.679488    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:26.381755 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:26.381719    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:27.178527 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:27.178460    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:27.873205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:27.873169    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:28.154511 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:28.154399    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:28.669413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:28.669380    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:29.405893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:29.405857    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:30.173553 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:30.173487    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:30.897220 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:30.897171    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:31.686331 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:31.686292    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:32.395850 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:32.395782    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:33.080885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:33.078593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:33.726981 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:33.726946    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:34.396108 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:34.396069    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:35.061839 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:35.061801    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:35.768554 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:35.768482    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:36.499069 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:36.499011    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:37.400728 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:37.400691    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:38.163187 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:38.163139    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:38.845966 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:38.845931    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:39.593582 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:39.593540    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:40.016268 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:48:40.016224    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:48:40.319212 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:40.319119    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:41.081589 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:41.081544    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:41.803167 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:41.803082    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:42.514631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:42.514594    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:43.751752 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:43.751702    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:44.517118 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:44.517077    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:45.219998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:45.219948    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:45.989020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:45.988981    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:46.694907 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:46.694836    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:47.423214 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:47.423174    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:48.258240 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:48.258200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:48.962984 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:48.962945    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:49.761205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:49.761169    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:50.475941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:50.475907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:51.168255 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:51.168214    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:51.513941 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:51.513901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:52.236111 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:52.236075    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:52.883401 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:52.883360    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:53.674207 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:53.674162    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:54.508669 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:54.508605    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:54.528836 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:54.528798    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:48:55.465128 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:55.465092    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:56.305225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:56.304332    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:57.037126 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:57.037089    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:57.795058 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:57.795022    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:58.780247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:58.780208    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:48:59.817603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:48:59.817565    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:00.663145 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:00.663105    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:01.461683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:01.461647    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:02.162637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:02.162599    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:02.685995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:02.685939    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:03.260947 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:03.260912    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:04.214264 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:04.214225    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:05.089031 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:05.088992    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:05.544867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:05.544816    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:49:05.834579 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:05.834446    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:06.774487 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:06.774447    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:07.481447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:07.481410    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:08.194078 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:08.194035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:09.083216 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:09.083181    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:09.909789 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:09.909747    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:10.015908 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:49:10.015868    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:49:10.719237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:10.719200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:11.578250 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:11.578168    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:12.270693 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:12.270650    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:13.106645 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:13.106606    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:13.413280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:13.413188    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:13.820156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:13.820123    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:14.184990 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:14.184855    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:14.973417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:14.973381    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:15.547633 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:15.547593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:15.695474 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:15.695433    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:15.840034 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:15.839944    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:15.994653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:15.994607    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:16.210413 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:16.210374    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:16.881697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:16.881656    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:17.370792 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:17.370752    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:17.562766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:17.562731    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:17.956642 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:17.956608    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:18.680107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:18.680063    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:18.937913 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:18.937821    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:49:19.572163 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:19.571065    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:20.274801 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:20.274763    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:21.092075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:21.092041    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:21.975526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:21.975477    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:22.677460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:22.677421    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:23.494309 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:23.494273    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:23.747017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:23.746908    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:23.873509 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:23.873444    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:24.014134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.014047    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"]
Nov 01 22:49:24.016891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.016864    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"]
Nov 01 22:49:24.017098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.017074    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy" containerID="cri-o://41341b41d98777c9ed47d6338d9a7b4dca17ea533770924644e76cc589097c17" gracePeriod=30
Nov 01 22:49:24.017208 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.017149    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" containerID="cri-o://3f1692904035236bb09599878660435a74d123611e3f6301bb80b77995635a2b" gracePeriod=30
Nov 01 22:49:24.044033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.043994    5057 patch_prober.go:29] interesting pod/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Nov 01 22:49:24.044230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.044102    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Nov 01 22:49:24.079641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.079604    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:24.375894 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:24.375794    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:25.396774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:25.396733    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:26.173147 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:26.173105    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:26.872953 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:26.872916    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:27.591452 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:27.591405    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:28.269885 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:28.269851    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:29.087107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.087072    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:29.200465 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.200425    5057 generic.go:356] "Generic (PLEG): container finished" podID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerID="3f1692904035236bb09599878660435a74d123611e3f6301bb80b77995635a2b" exitCode=0
Nov 01 22:49:29.215821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.207285    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" event={"ID":"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1","Type":"ContainerDied","Data":"3f1692904035236bb09599878660435a74d123611e3f6301bb80b77995635a2b"}
Nov 01 22:49:29.218961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.218931    5057 generic.go:356] "Generic (PLEG): container finished" podID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerID="41341b41d98777c9ed47d6338d9a7b4dca17ea533770924644e76cc589097c17" exitCode=0
Nov 01 22:49:29.219104 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.218994    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" event={"ID":"afbfc2e3-7c03-40b2-a82b-be84c738f252","Type":"ContainerDied","Data":"41341b41d98777c9ed47d6338d9a7b4dca17ea533770924644e76cc589097c17"}
Nov 01 22:49:29.301574 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.301460    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:49:29.345882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.345850    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:49:29.408005 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.407967    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408059    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408105    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408205 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408146    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408235    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408293    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408331    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408365    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408412 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408401    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408431    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408474    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408554    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408591    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408623    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408673    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408715    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408740    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket\") pod \"afbfc2e3-7c03-40b2-a82b-be84c738f252\" (UID: \"afbfc2e3-7c03-40b2-a82b-be84c738f252\") "
Nov 01 22:49:29.408782 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.408772    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert\") pod \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\" (UID: \"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1\") "
Nov 01 22:49:29.409397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.409370    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.409483 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.409462    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:49:29.409793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.409771    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:49:29.410035 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.410018    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.410194 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.410146    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.412714    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "kube-api-access-v56kn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.412919    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.412950    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.413161    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.413344    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.413547    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.413760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.413725    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.416188 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.416153    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.416438 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.416400    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "kube-api-access-d4lm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:49:29.416687 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.416665    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:29.417377 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.417349    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:49:29.417870 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.417846    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token" pod "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" (UID: "6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:49:29.417975 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.417884    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo" pod "afbfc2e3-7c03-40b2-a82b-be84c738f252" (UID: "afbfc2e3-7c03-40b2-a82b-be84c738f252"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:49:29.510916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510880    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.510916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510911    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.510916 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510920    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510928    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510937    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510946    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510955    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510963    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v56kn\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-kube-api-access-v56kn\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510973    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510981    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.510998    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511007    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511015    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4lm6\" (UniqueName: \"kubernetes.io/projected/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-kube-api-access-d4lm6\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511024    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/afbfc2e3-7c03-40b2-a82b-be84c738f252-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511032    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511041    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511050    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.511176 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.511058    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/afbfc2e3-7c03-40b2-a82b-be84c738f252-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:29.831025 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:29.830965    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:30.226748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.226716    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"
Nov 01 22:49:30.226957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.226745    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm" event={"ID":"afbfc2e3-7c03-40b2-a82b-be84c738f252","Type":"ContainerDied","Data":"1cb79489385daa0dde9a96bd43b770bf07e5b591d61216197a9702df59c4037d"}
Nov 01 22:49:30.226957 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.226804    5057 scope.go:122] "RemoveContainer" containerID="41341b41d98777c9ed47d6338d9a7b4dca17ea533770924644e76cc589097c17"
Nov 01 22:49:30.228394 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.228368    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p" event={"ID":"6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1","Type":"ContainerDied","Data":"7fe942d97a5b0ffd8ae33b334c1d84e891b099cb3c64af652cc66bb1c368da0d"}
Nov 01 22:49:30.228525 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.228375    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"
Nov 01 22:49:30.237572 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.237549    5057 scope.go:122] "RemoveContainer" containerID="3f1692904035236bb09599878660435a74d123611e3f6301bb80b77995635a2b"
Nov 01 22:49:30.260329 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.260297    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"]
Nov 01 22:49:30.272706 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.272675    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-fg82p"]
Nov 01 22:49:30.307267 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.307232    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"]
Nov 01 22:49:30.322011 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.321979    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1fddbd-openshift-default-689959cbb5-t5pnm"]
Nov 01 22:49:30.592375 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:30.592281    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:31.269400 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:31.269360    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:31.396428 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:31.396392    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" path="/var/lib/kubelet/pods/6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1/volumes"
Nov 01 22:49:31.396917 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:31.396902    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" path="/var/lib/kubelet/pods/afbfc2e3-7c03-40b2-a82b-be84c738f252/volumes"
Nov 01 22:49:32.007835 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:32.007796    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:32.697703 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:32.697655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:33.489766 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:33.489732    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:34.189183 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:34.189132    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:34.798664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:34.798624    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:34.954811 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:34.954756    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:35.594345 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:35.594118    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:36.215064 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:36.215024    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:36.596311 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:36.596200    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:36.769733 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:36.769687    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:37.090963 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:37.090912    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:37.566891 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:37.566840    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:38.308097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:38.308059    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:39.081215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:39.081172    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:39.422722 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:39.422527    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:49:39.983020 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:39.982962    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:40.015553 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:49:40.015512    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:49:40.909763 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:40.909729    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:41.905272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:41.905232    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:42.675195 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:42.675155    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:43.753623 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:43.753582    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:44.488343 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:44.488301    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:45.291109 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:45.291072    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:45.793724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:45.793680    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:46.012227 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:46.012187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:46.116048 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:46.115952    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"]
Nov 01 22:49:46.116238 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:46.116206    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy" containerID="cri-o://88c4294d47a87e6eeee3959a0fd5a846972f040672747435dbcd477393c0fe8b" gracePeriod=30
Nov 01 22:49:46.203631 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:46.203593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:47.123670 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:47.123632    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:47.831174 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:47.831125    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:47.985883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:47.985847    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:48.888435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:48.888388    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:49.661628 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:49.661590    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:50.394653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:50.394608    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:51.097995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.097937    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:51.352840 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.352759    5057 generic.go:356] "Generic (PLEG): container finished" podID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerID="88c4294d47a87e6eeee3959a0fd5a846972f040672747435dbcd477393c0fe8b" exitCode=0
Nov 01 22:49:51.353000 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.352830    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" event={"ID":"a1d8dc09-b7c3-434a-9595-d50e5cb06f77","Type":"ContainerDied","Data":"88c4294d47a87e6eeee3959a0fd5a846972f040672747435dbcd477393c0fe8b"}
Nov 01 22:49:51.383697 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.383668    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:49:51.461940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.461903    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.461950    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.461983    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462000    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462028    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462054    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462091    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462124    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462203    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo\") pod \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\" (UID: \"a1d8dc09-b7c3-434a-9595-d50e5cb06f77\") "
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462390    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:51.462472 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462452    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:49:51.463098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462732    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:51.463098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462753    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:51.463098 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.462788    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:51.463443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.463411    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.463443 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.463436    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.463592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.463449    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.463592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.463462    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.463592 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.463474    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.464522 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.464475    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:49:51.464617 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.464587    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:49:51.464796 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.464778    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:49:51.465133 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.465114    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5" pod "a1d8dc09-b7c3-434a-9595-d50e5cb06f77" (UID: "a1d8dc09-b7c3-434a-9595-d50e5cb06f77"). InnerVolumeSpecName "kube-api-access-64qk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:49:51.564893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.564853    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.564893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.564891    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.564893 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.564902    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64qk5\" (UniqueName: \"kubernetes.io/projected/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-kube-api-access-64qk5\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.565138 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.564913    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d8dc09-b7c3-434a-9595-d50e5cb06f77-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:49:51.783806 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:51.783757    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:52.360178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:52.360145    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"
Nov 01 22:49:52.360178 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:52.360157    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq" event={"ID":"a1d8dc09-b7c3-434a-9595-d50e5cb06f77","Type":"ContainerDied","Data":"68fe5c38d6fe9eeb526d3d954f7ce581f7763cf9ae3a47ad689ed63d28a8be82"}
Nov 01 22:49:52.360469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:52.360257    5057 scope.go:122] "RemoveContainer" containerID="88c4294d47a87e6eeee3959a0fd5a846972f040672747435dbcd477393c0fe8b"
Nov 01 22:49:52.383101 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:52.383071    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"]
Nov 01 22:49:52.389299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:52.389270    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9717e2-openshift-default-556879b8bc-45jpq"]
Nov 01 22:49:52.457168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:52.457130    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:53.130630 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:53.130593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:53.398288 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:53.398205    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" path="/var/lib/kubelet/pods/a1d8dc09-b7c3-434a-9595-d50e5cb06f77/volumes"
Nov 01 22:49:53.794272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:53.794234    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:54.488116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:54.488076    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:55.296632 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:55.296600    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:56.077980 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:56.077942    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:56.857181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:56.857144    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:56.989794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:56.989759    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:57.490603 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:57.490562    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:58.281821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:58.281770    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:58.930747 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:58.930706    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:59.141272 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:59.141237    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:59.294010 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:59.293976    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:59.466660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:59.466622    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:49:59.758138 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:49:59.758082    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:00.446791 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:00.446754    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:01.207371 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:01.207327    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:01.706277 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:01.706241    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:50:01.921665 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:01.921626    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:02.692521 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:02.692457    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:03.426903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:03.426869    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:04.077478 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:04.077436    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:04.547577 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:04.547526    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:04.775107 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:04.775068    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:04.937201 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:04.937104    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:05.462481 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:05.462440    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:06.438641 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:06.438606    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:06.667514 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:06.667445    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:06.808009 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:06.807971    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:07.295444 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:07.295389    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:08.193011 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:08.192977    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:08.887807 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:08.887772    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:09.605895 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:09.605861    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:10.015243 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:50:10.015205    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:50:10.344861 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:10.344765    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:10.977376 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:10.977336    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:11.814321 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:11.814283    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:12.488531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:12.488453    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:13.183116 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:13.183073    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:13.725793 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:13.725757    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:13.891919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:13.891862    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:14.551330 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:14.551286    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:15.269013 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:15.268970    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:15.960859 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:15.960827    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:16.682398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:16.682348    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:17.415273 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:17.415238    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:17.568022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:17.567984    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:17.843734 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:17.843693    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:18.124590 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:18.124470    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:18.377866 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:18.377767    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:18.586601 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:18.586561    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:18.939223 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:18.939184    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:19.293653 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:19.293610    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:20.111130 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:20.109972    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:20.692565 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:20.692530    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:50:20.855560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:20.855522    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:21.804813 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:21.804775    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:22.521955 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:22.521919    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:23.282686 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:23.282650    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:24.091872 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:24.091844    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:24.782976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:24.782939    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:25.552007 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:25.551965    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:25.730399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:25.730357    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:25.911883 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:25.911782    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:26.369905 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:26.369821    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:26.536226 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:26.536171    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:27.287903 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:27.287865    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:28.093542 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:28.093468    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:28.809639 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:28.809607    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:29.486173 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:29.486138    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:30.284559 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:30.284523    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:30.563856 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:30.563762    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:30.722808 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:30.722775    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:31.027664 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:31.027619    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:31.292606 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:31.292486    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"]
Nov 01 22:50:31.449181 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:31.449143    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:31.885753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:31.885715    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:32.791466 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:32.791424    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:33.474417 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:33.474377    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:34.086210 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:34.086172    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:34.785124 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:34.785076    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:35.657735 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:35.657700    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:36.475940 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:36.475899    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:36.734422 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:36.734336    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:36.880652 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:36.880613    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:40.015554 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:50:40.015489    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:50:46.313587 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.313521    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main" containerID="cri-o://c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e" gracePeriod=45
Nov 01 22:50:46.571855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.571788    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:50:46.663044 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663006    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn\" (UniqueName: \"kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn\") pod \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") "
Nov 01 22:50:46.663237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663087    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs\" (UniqueName: \"kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs\") pod \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") "
Nov 01 22:50:46.663237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663125    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache\") pod \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") "
Nov 01 22:50:46.663237 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663175    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home\") pod \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") "
Nov 01 22:50:46.663415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663281    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir\") pod \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") "
Nov 01 22:50:46.663415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663323    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm\") pod \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\" (UID: \"d2c13e54-c14f-46c5-bd5d-03414f2534b1\") "
Nov 01 22:50:46.663531 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663435    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache" pod "d2c13e54-c14f-46c5-bd5d-03414f2534b1" (UID: "d2c13e54-c14f-46c5-bd5d-03414f2534b1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:50:46.663929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663645    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home" pod "d2c13e54-c14f-46c5-bd5d-03414f2534b1" (UID: "d2c13e54-c14f-46c5-bd5d-03414f2534b1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:50:46.663929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663668    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir" pod "d2c13e54-c14f-46c5-bd5d-03414f2534b1" (UID: "d2c13e54-c14f-46c5-bd5d-03414f2534b1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:50:46.663929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663734    5057 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-model-cache\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:46.663929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663750    5057 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-home\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:46.663929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.663762    5057 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tmp-dir\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:46.665800 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.665766    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs" pod "d2c13e54-c14f-46c5-bd5d-03414f2534b1" (UID: "d2c13e54-c14f-46c5-bd5d-03414f2534b1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:50:46.665911 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.665792    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm" pod "d2c13e54-c14f-46c5-bd5d-03414f2534b1" (UID: "d2c13e54-c14f-46c5-bd5d-03414f2534b1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:50:46.665960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.665925    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn" pod "d2c13e54-c14f-46c5-bd5d-03414f2534b1" (UID: "d2c13e54-c14f-46c5-bd5d-03414f2534b1"). InnerVolumeSpecName "kube-api-access-x2hfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:50:46.676096 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.676061    5057 generic.go:356] "Generic (PLEG): container finished" podID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerID="c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e" exitCode=0
Nov 01 22:50:46.676242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.676108    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"
Nov 01 22:50:46.676242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.676201    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" event={"ID":"d2c13e54-c14f-46c5-bd5d-03414f2534b1","Type":"ContainerDied","Data":"c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e"}
Nov 01 22:50:46.676242 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.676237    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd" event={"ID":"d2c13e54-c14f-46c5-bd5d-03414f2534b1","Type":"ContainerDied","Data":"6408c67e35cc1fb29313da758b8b54e160ca5837fb982436dfbb83ceaaf0792d"}
Nov 01 22:50:46.676430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.676260    5057 scope.go:122] "RemoveContainer" containerID="c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e"
Nov 01 22:50:46.697225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.697200    5057 scope.go:122] "RemoveContainer" containerID="c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e"
Nov 01 22:50:46.697638 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:50:46.697615    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e\": container with ID starting with c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e not found: ID does not exist" containerID="c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e"
Nov 01 22:50:46.697724 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.697650    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e"} err="failed to get container status \"c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e\": rpc error: code = NotFound desc = could not find container \"c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e\": container with ID starting with c0ccf09aea13792f720d5fc28fadeee333904b308129d9e5a6858db9e0d68f9e not found: ID does not exist"
Nov 01 22:50:46.710186 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.710112    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"]
Nov 01 22:50:46.712915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.712884    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-aa441b/distinct-w5-aa441b-kserve-5456b84c89-8dtnd"]
Nov 01 22:50:46.764307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.764276    5057 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2c13e54-c14f-46c5-bd5d-03414f2534b1-dshm\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:46.764307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.764304    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2hfn\" (UniqueName: \"kubernetes.io/projected/d2c13e54-c14f-46c5-bd5d-03414f2534b1-kube-api-access-x2hfn\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:46.764526 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:46.764317    5057 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2c13e54-c14f-46c5-bd5d-03414f2534b1-tls-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:47.395822 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:47.395785    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" path="/var/lib/kubelet/pods/d2c13e54-c14f-46c5-bd5d-03414f2534b1/volumes"
Nov 01 22:50:48.419773 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:48.419732    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:50:52.062591 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.062545    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"]
Nov 01 22:50:52.063099 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.062835    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg" podUID="5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" containerName="maas-api" containerID="cri-o://c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db" gracePeriod=30
Nov 01 22:50:52.172882 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.172843    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:52.357783 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.357754    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:50:52.427774 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.427735    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:52.429683 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.429651    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls\") pod \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") "
Nov 01 22:50:52.429830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.429741    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5\" (UniqueName: \"kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5\") pod \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") "
Nov 01 22:50:52.429830 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.429795    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca\") pod \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") "
Nov 01 22:50:52.429945 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.429833    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs\") pod \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\" (UID: \"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8\") "
Nov 01 22:50:52.430397 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.430369    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca" pod "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" (UID: "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:50:52.432084 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.432059    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls" pod "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" (UID: "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:50:52.432213 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.432194    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs" pod "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" (UID: "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:50:52.432291 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.432249    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5" pod "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" (UID: "5ef56ea1-0f1e-4553-8f34-b2a508dfcff8"). InnerVolumeSpecName "kube-api-access-bklc5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:50:52.531215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.531173    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:52.531215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.531207    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:52.531215 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.531220    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bklc5\" (UniqueName: \"kubernetes.io/projected/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-kube-api-access-bklc5\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:52.531463 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.531235    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:50:52.575196 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.575151    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:52.708672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.708635    5057 generic.go:356] "Generic (PLEG): container finished" podID="5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" containerID="c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db" exitCode=0
Nov 01 22:50:52.708884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.708680    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"
Nov 01 22:50:52.708884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.708717    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg" event={"ID":"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8","Type":"ContainerDied","Data":"c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db"}
Nov 01 22:50:52.708884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.708769    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg" event={"ID":"5ef56ea1-0f1e-4553-8f34-b2a508dfcff8","Type":"ContainerDied","Data":"47d5960a44dcde6e3def487c141a40ae3376f07e6bb2395fa3fb06652c946a39"}
Nov 01 22:50:52.708884 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.708797    5057 scope.go:122] "RemoveContainer" containerID="c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db"
Nov 01 22:50:52.719995 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.719966    5057 scope.go:122] "RemoveContainer" containerID="c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db"
Nov 01 22:50:52.720356 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:50:52.720336    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db\": container with ID starting with c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db not found: ID does not exist" containerID="c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db"
Nov 01 22:50:52.720442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.720363    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db"} err="failed to get container status \"c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db\": rpc error: code = NotFound desc = could not find container \"c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db\": container with ID starting with c5da9ff6eb2b8bf88db056ecee32edb61d932cf582629a19cafb99573ebdd3db not found: ID does not exist"
Nov 01 22:50:52.732955 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.732930    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"]
Nov 01 22:50:52.746144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.746115    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-aa441b-5cf5465957-kjfpg"]
Nov 01 22:50:52.976986 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:52.976895    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:53.396678 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:53.396599    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" path="/var/lib/kubelet/pods/5ef56ea1-0f1e-4553-8f34-b2a508dfcff8/volumes"
Nov 01 22:50:54.034243 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:54.034208    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:50:55.235760 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:55.235716    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:55.395047 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:55.395000    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:59.222068 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.222035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:59.362462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.362424    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:50:59.509896 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.509824    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"]
Nov 01 22:50:59.510070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.510033    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy" containerID="cri-o://5eb7672754f6255ed3a3c1c143773e5663da15479ed124fc7787c8942aef6353" gracePeriod=30
Nov 01 22:50:59.524732 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.524708    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"]
Nov 01 22:50:59.524914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.524888    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" containerID="cri-o://5d8f2f116104ed5228cf1488be54754faee08fee77d47ffb0d59e46121848566" gracePeriod=30
Nov 01 22:50:59.588218 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:50:59.588187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:02.056757 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.056719    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b57d74f86-p96tx"]
Nov 01 22:51:02.057260 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.056992    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-7b57d74f86-p96tx" podUID="a741756c-9713-4bad-981f-0f1b25ffa080" containerName="manager" containerID="cri-o://b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0" gracePeriod=30
Nov 01 22:51:02.312480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.312407    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:51:02.433636 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.433600    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x\" (UniqueName: \"kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x\") pod \"a741756c-9713-4bad-981f-0f1b25ffa080\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") "
Nov 01 22:51:02.433826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.433642    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert\") pod \"a741756c-9713-4bad-981f-0f1b25ffa080\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") "
Nov 01 22:51:02.433826 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.433688    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs\") pod \"a741756c-9713-4bad-981f-0f1b25ffa080\" (UID: \"a741756c-9713-4bad-981f-0f1b25ffa080\") "
Nov 01 22:51:02.435961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.435925    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert" pod "a741756c-9713-4bad-981f-0f1b25ffa080" (UID: "a741756c-9713-4bad-981f-0f1b25ffa080"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:51:02.435961 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.435940    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x" pod "a741756c-9713-4bad-981f-0f1b25ffa080" (UID: "a741756c-9713-4bad-981f-0f1b25ffa080"). InnerVolumeSpecName "kube-api-access-vrl4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:51:02.436144 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.436126    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs" pod "a741756c-9713-4bad-981f-0f1b25ffa080" (UID: "a741756c-9713-4bad-981f-0f1b25ffa080"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:51:02.535168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.535131    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrl4x\" (UniqueName: \"kubernetes.io/projected/a741756c-9713-4bad-981f-0f1b25ffa080-kube-api-access-vrl4x\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:02.535168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.535164    5057 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-webhook-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:02.535168 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.535175    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a741756c-9713-4bad-981f-0f1b25ffa080-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:02.766635 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.766598    5057 generic.go:356] "Generic (PLEG): container finished" podID="a741756c-9713-4bad-981f-0f1b25ffa080" containerID="b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0" exitCode=0
Nov 01 22:51:02.766833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.766653    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7b57d74f86-p96tx"
Nov 01 22:51:02.766833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.766679    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b57d74f86-p96tx" event={"ID":"a741756c-9713-4bad-981f-0f1b25ffa080","Type":"ContainerDied","Data":"b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0"}
Nov 01 22:51:02.766833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.766733    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7b57d74f86-p96tx" event={"ID":"a741756c-9713-4bad-981f-0f1b25ffa080","Type":"ContainerDied","Data":"ef55059d421e095467179f12fb7474da11c09bb50a3aa192be6acf658f4c86df"}
Nov 01 22:51:02.766833 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.766755    5057 scope.go:122] "RemoveContainer" containerID="b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0"
Nov 01 22:51:02.777408 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.777390    5057 scope.go:122] "RemoveContainer" containerID="b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0"
Nov 01 22:51:02.777745 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:51:02.777725    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0\": container with ID starting with b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0 not found: ID does not exist" containerID="b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0"
Nov 01 22:51:02.777820 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.777756    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0"} err="failed to get container status \"b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0\": rpc error: code = NotFound desc = could not find container \"b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0\": container with ID starting with b717d75321f5405fab5382fa35ff58a418e9dbde798936bae462ee00ccc42ed0 not found: ID does not exist"
Nov 01 22:51:02.796837 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.796810    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7b57d74f86-p96tx"]
Nov 01 22:51:02.802155 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:02.802132    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7b57d74f86-p96tx"]
Nov 01 22:51:03.401479 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:03.401084    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a741756c-9713-4bad-981f-0f1b25ffa080" path="/var/lib/kubelet/pods/a741756c-9713-4bad-981f-0f1b25ffa080/volumes"
Nov 01 22:51:04.799598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:04.799386    5057 generic.go:356] "Generic (PLEG): container finished" podID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerID="5eb7672754f6255ed3a3c1c143773e5663da15479ed124fc7787c8942aef6353" exitCode=0
Nov 01 22:51:04.799598 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:04.799526    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" event={"ID":"94b34e89-e83e-406d-8c6b-8589cbd6fd63","Type":"ContainerDied","Data":"5eb7672754f6255ed3a3c1c143773e5663da15479ed124fc7787c8942aef6353"}
Nov 01 22:51:04.801997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:04.801886    5057 generic.go:356] "Generic (PLEG): container finished" podID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerID="5d8f2f116104ed5228cf1488be54754faee08fee77d47ffb0d59e46121848566" exitCode=0
Nov 01 22:51:04.801997 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:04.801940    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" event={"ID":"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6","Type":"ContainerDied","Data":"5d8f2f116104ed5228cf1488be54754faee08fee77d47ffb0d59e46121848566"}
Nov 01 22:51:04.968008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:04.967633    5057 patch_prober.go:29] interesting pod/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Nov 01 22:51:04.968008 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:04.967712    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Nov 01 22:51:05.105562 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.099962    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:51:05.210241 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.209903    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210241 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210030    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210474 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210450    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210602 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210565    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210672 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210612    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210729 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210684    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210788 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210735    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210849 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210799    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.210919 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.210871    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy\") pod \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\" (UID: \"94b34e89-e83e-406d-8c6b-8589cbd6fd63\") "
Nov 01 22:51:05.211307 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.211180    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.211998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.211958    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.212134 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.212014    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.212430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.212409    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.212850 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.212823    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:51:05.213748 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.213725    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.213998 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.213979    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:51:05.214097 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.214070    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "kube-api-access-5kpdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:51:05.214366 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.214351    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token" pod "94b34e89-e83e-406d-8c6b-8589cbd6fd63" (UID: "94b34e89-e83e-406d-8c6b-8589cbd6fd63"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:51:05.307618 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.307587    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:51:05.311654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311632    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311654 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311652    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311664    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kpdf\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-kube-api-access-5kpdf\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311674    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311684    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311693    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311703    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311711    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.311831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.311721    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94b34e89-e83e-406d-8c6b-8589cbd6fd63-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.412211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412135    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412211 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412180    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412220    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412269    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412309    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412328    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412368    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412410    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412447 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412428    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket\") pod \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\" (UID: \"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6\") "
Nov 01 22:51:05.412864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412737    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.412864 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412828    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:51:05.412971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412913    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.412971 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.412950    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.413247 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.413225    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.415456 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.415422    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:51:05.415701 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.415678    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:51:05.416190 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.416150    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 22:51:05.417191 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.417165    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97" pod "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" (UID: "fe9cbfb3-42ea-4b6e-a373-699f5daed5f6"). InnerVolumeSpecName "kube-api-access-bsx97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:51:05.514094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514062    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514091    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514094 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514101    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514110    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514121    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514131    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514139    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514147    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.514337 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.514155    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsx97\" (UniqueName: \"kubernetes.io/projected/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6-kube-api-access-bsx97\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:51:05.807968 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.807935    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"
Nov 01 22:51:05.808480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.807965    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq" event={"ID":"94b34e89-e83e-406d-8c6b-8589cbd6fd63","Type":"ContainerDied","Data":"02cfef4fc38f7601917746322fb52e31d406bed8627365c2b1be185d65cbcfbc"}
Nov 01 22:51:05.808480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.808037    5057 scope.go:122] "RemoveContainer" containerID="5eb7672754f6255ed3a3c1c143773e5663da15479ed124fc7787c8942aef6353"
Nov 01 22:51:05.809480 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.809455    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c" event={"ID":"fe9cbfb3-42ea-4b6e-a373-699f5daed5f6","Type":"ContainerDied","Data":"472e99993a11170a6d0288884e565d0cebfc909cecbf96cefce8f863baafc601"}
Nov 01 22:51:05.809611 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.809566    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"
Nov 01 22:51:05.819784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.819769    5057 scope.go:122] "RemoveContainer" containerID="5d8f2f116104ed5228cf1488be54754faee08fee77d47ffb0d59e46121848566"
Nov 01 22:51:05.826609 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.826585    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"]
Nov 01 22:51:05.832753 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.832728    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-vxwlq"]
Nov 01 22:51:05.849033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.849004    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"]
Nov 01 22:51:05.852977 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:05.852952    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-aa441b-openshift-default-59698dbb8f-zc58c"]
Nov 01 22:51:07.397360 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:07.397326    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" path="/var/lib/kubelet/pods/94b34e89-e83e-406d-8c6b-8589cbd6fd63/volumes"
Nov 01 22:51:07.397831 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:07.397816    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" path="/var/lib/kubelet/pods/fe9cbfb3-42ea-4b6e-a373-699f5daed5f6/volumes"
Nov 01 22:51:09.127914 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:09.127876    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:09.481731 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:09.481699    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:09.707777 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:09.707740    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:10.014454 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:51:10.014412    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:51:13.751236 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:13.751194    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:15.442460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:15.442423    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:18.009855 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:18.009821    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:20.816915 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:20.816877    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:20.931442 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:20.931405    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:36.975657 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:36.975572    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:39.232474 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:39.232386    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:39.386299 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:39.386265    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:39.556225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:39.556143    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:40.013157 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:51:40.013116    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:51:47.285677 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:47.285640    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:51:47.471578 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:51:47.471539    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:52:10.013229 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:52:10.013188    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:52:11.896075 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:11.896033    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:52:13.875619 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:13.875562    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:52:20.178921 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.178883    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"]
Nov 01 22:52:20.179506 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.179108    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm" podUID="619f00b4-a838-4c64-96e4-7e1fcb83d1fe" containerName="discovery" containerID="cri-o://9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8" gracePeriod=30
Nov 01 22:52:20.435285 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.435225    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:52:20.564502 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564448    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564548    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564577    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564607    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564679 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564639    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564679    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564716    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig\") pod \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\" (UID: \"619f00b4-a838-4c64-96e4-7e1fcb83d1fe\") "
Nov 01 22:52:20.564902 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.564874    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 22:52:20.565529 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.565502    5057 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-ca-configmap\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:20.566994 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.566967    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:52:20.567156 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.567128    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "kube-api-access-84jwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:52:20.567308 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.567281    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 22:52:20.567415 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.567388    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:52:20.567460 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.567407    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 22:52:20.567794 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.567772    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts" pod "619f00b4-a838-4c64-96e4-7e1fcb83d1fe" (UID: "619f00b4-a838-4c64-96e4-7e1fcb83d1fe"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 22:52:20.666274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.666240    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84jwj\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-kube-api-access-84jwj\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:20.666274 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.666271    5057 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-cacerts\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:20.666469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.666282    5057 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-local-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:20.666469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.666292    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:20.666469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.666300    5057 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-csr-dns-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:20.666469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:20.666308    5057 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/619f00b4-a838-4c64-96e4-7e1fcb83d1fe-istio-kubeconfig\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 22:52:21.205960 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.205922    5057 generic.go:356] "Generic (PLEG): container finished" podID="619f00b4-a838-4c64-96e4-7e1fcb83d1fe" containerID="9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8" exitCode=0
Nov 01 22:52:21.206399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.205974    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"
Nov 01 22:52:21.206399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.205997    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm" event={"ID":"619f00b4-a838-4c64-96e4-7e1fcb83d1fe","Type":"ContainerDied","Data":"9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8"}
Nov 01 22:52:21.206399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.206050    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm" event={"ID":"619f00b4-a838-4c64-96e4-7e1fcb83d1fe","Type":"ContainerDied","Data":"a4fe23bd61d9ab7f3260f08e86b10b116127aa94a5db5e8a5cab980b862cc6ff"}
Nov 01 22:52:21.206399 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.206069    5057 scope.go:122] "RemoveContainer" containerID="9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8"
Nov 01 22:52:21.216740 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.216724    5057 scope.go:122] "RemoveContainer" containerID="9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8"
Nov 01 22:52:21.216975 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:52:21.216955    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8\": container with ID starting with 9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8 not found: ID does not exist" containerID="9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8"
Nov 01 22:52:21.217017 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.216990    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8"} err="failed to get container status \"9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8\": rpc error: code = NotFound desc = could not find container \"9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8\": container with ID starting with 9dfbfbe0b633b4973c92915a387f21454bdbd56b28e4c2bfb9c6a0612dada0b8 not found: ID does not exist"
Nov 01 22:52:21.228532 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.228508    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"]
Nov 01 22:52:21.232637 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.232618    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-246qm"]
Nov 01 22:52:21.396784 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:21.396754    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="619f00b4-a838-4c64-96e4-7e1fcb83d1fe" path="/var/lib/kubelet/pods/619f00b4-a838-4c64-96e4-7e1fcb83d1fe/volumes"
Nov 01 22:52:23.729430 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:23.729405    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:52:23.739225 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:23.739199    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:52:27.498028 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:27.497990    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:52:31.235536 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:31.235478    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:52:37.791674 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:37.791632    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:52:40.013809 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:52:40.013770    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:52:46.173821 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:46.173774    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:52:49.685516 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:49.684469    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:52:49.876523 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:49.876049    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:52:58.929505 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:52:58.929452    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:53:10.013176 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:53:10.013139    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:53:13.950453 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:13.950418    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:53:14.128398 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:14.128365    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:53:15.269342 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:15.269306    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:53:23.183280 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:23.183247    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:53:33.395761 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:33.395731    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:53:37.736541 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:37.728530    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:53:40.014056 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:53:40.014016    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:53:45.704138 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:45.704104    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:53:45.860934 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:45.860898    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:53:53.992462 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:53.992424    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:53:57.477518 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:53:57.476447    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:54:05.286713 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:05.286676    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:54:05.422022 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:05.421969    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:54:10.012865 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:54:10.012827    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:54:12.013402 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:12.013369    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:54:14.031558 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:14.031522    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:54:14.177976 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:14.177939    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:54:17.128867 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:17.128828    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:54:30.727033 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:30.726985    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:54:34.259469 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:34.259351    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:54:40.013615 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:54:40.013579    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:54:48.534688 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:48.534651    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:54:59.125127 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:54:59.125091    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:55:03.303732 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:03.301704    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:55:10.013810 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:55:10.013770    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:55:14.029070 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:14.029031    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:55:14.209159 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:14.209066    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:55:17.603209 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:17.603171    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:55:21.230435 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:21.230392    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:55:27.216271 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:27.216224    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:55:40.014319 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:55:40.014278    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:55:45.417560 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:45.417522    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:55:55.782570 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:55:55.782536    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 22:56:10.013180 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:56:10.013145    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:56:14.112901 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:56:14.112859    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:56:14.266776 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:56:14.266680    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:56:40.014509 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:56:40.014452    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:57:10.014321 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:57:10.014257    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:57:14.174414 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:57:14.174370    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:57:14.329660 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:57:14.329624    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:57:23.759929 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:57:23.759898    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:57:23.770418 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:57:23.770393    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 22:57:40.014149 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:57:40.014065    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:58:10.013527 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:58:10.013466    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:58:14.150557 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:58:14.150518    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:58:18.463369 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:58:18.463335    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:58:40.015043 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:58:40.014996    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:59:10.014952 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:59:10.014914    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 22:59:14.198230 ip-10-0-141-148 kubenswrapper[5057]: I1101 22:59:14.198188    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 22:59:40.014640 ip-10-0-141-148 kubenswrapper[5057]: E1101 22:59:40.014523    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:00:10.014095 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:00:10.014049    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:00:14.245438 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:00:14.245401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:00:40.013509 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:00:40.013417    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:00:41.265149 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:00:41.265114    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:10.013124 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:01:10.013086    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:01:10.378747 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:10.378654    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"]
Nov 01 23:01:10.378915 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:10.378874    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy" containerID="cri-o://95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6" gracePeriod=30
Nov 01 23:01:10.380537 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:10.380511    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"]
Nov 01 23:01:10.380734 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:10.380699    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy" containerID="cri-o://b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7" gracePeriod=30
Nov 01 23:01:12.899726 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:12.899687    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:01:14.396547 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:14.396504    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:15.654508 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.654467    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 23:01:15.702763 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.702736    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 23:01:15.751042 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.750866    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751237 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751102    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751237 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751142    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751237 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751206    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751419 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751261    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751419 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751258    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.751419 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751315    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751419 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751342    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751667 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751428    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.751667 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.751559    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy\") pod \"58ff371d-d2ed-415c-988f-cffea5d08b83\" (UID: \"58ff371d-d2ed-415c-988f-cffea5d08b83\") "
Nov 01 23:01:15.752094 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.752063    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.752219 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.752117    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.752291 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.752272    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.752742 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.752704    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:01:15.753180 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.753146    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.754900 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.754850    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:01:15.755171 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.755144    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 23:01:15.755874 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.755847    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.755874 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.755866    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88" pod "58ff371d-d2ed-415c-988f-cffea5d08b83" (UID: "58ff371d-d2ed-415c-988f-cffea5d08b83"). InnerVolumeSpecName "kube-api-access-tcs88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:01:15.853400 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853360    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853400 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853408    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853700 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853428    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853700 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853467    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853700 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853544    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853700 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853577    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853700 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853659    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853708    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853756    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert\") pod \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\" (UID: \"75a10a58-ea8f-437c-bb3e-cd6347d6bf69\") "
Nov 01 23:01:15.853962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853774    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.853962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853847    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.853962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.853859    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854014    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854108    5057 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-data\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854130    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/58ff371d-d2ed-415c-988f-cffea5d08b83-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854145    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854159    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854172    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854185    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854225    5057 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854677 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854241    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/58ff371d-d2ed-415c-988f-cffea5d08b83-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854677 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854254    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854677 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854268    5057 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/58ff371d-d2ed-415c-988f-cffea5d08b83-workload-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854677 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854282    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tcs88\" (UniqueName: \"kubernetes.io/projected/58ff371d-d2ed-415c-988f-cffea5d08b83-kube-api-access-tcs88\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.854677 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854293    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:01:15.854677 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.854296    5057 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-credential-socket\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.856111 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.856084    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "kube-api-access-d9sb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:01:15.856111 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.856088    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Nov 01 23:01:15.856382 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.856362    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:01:15.856435 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.856366    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy" pod "75a10a58-ea8f-437c-bb3e-cd6347d6bf69" (UID: "75a10a58-ea8f-437c-bb3e-cd6347d6bf69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:01:15.954797 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.954759    5057 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-podinfo\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.954797 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.954795    5057 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-envoy\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.955032 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.954808    5057 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istio-token\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.955032 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.954821    5057 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-istiod-ca-cert\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.955032 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.954837    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9sb4\" (UniqueName: \"kubernetes.io/projected/75a10a58-ea8f-437c-bb3e-cd6347d6bf69-kube-api-access-d9sb4\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:01:15.963941 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.963912    5057 generic.go:356] "Generic (PLEG): container finished" podID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerID="b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7" exitCode=0
Nov 01 23:01:15.964092 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.963959    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"
Nov 01 23:01:15.964092 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.964001    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" event={"ID":"58ff371d-d2ed-415c-988f-cffea5d08b83","Type":"ContainerDied","Data":"b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7"}
Nov 01 23:01:15.964092 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.964061    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5" event={"ID":"58ff371d-d2ed-415c-988f-cffea5d08b83","Type":"ContainerDied","Data":"77f92e4f2db7fa78a758619441a9b5169ff19c0ce2214297828f695567e05a5d"}
Nov 01 23:01:15.964229 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.964093    5057 scope.go:122] "RemoveContainer" containerID="b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7"
Nov 01 23:01:15.965626 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.965601    5057 generic.go:356] "Generic (PLEG): container finished" podID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerID="95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6" exitCode=0
Nov 01 23:01:15.965758 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.965646    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"
Nov 01 23:01:15.965758 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.965677    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" event={"ID":"75a10a58-ea8f-437c-bb3e-cd6347d6bf69","Type":"ContainerDied","Data":"95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6"}
Nov 01 23:01:15.965758 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.965715    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl" event={"ID":"75a10a58-ea8f-437c-bb3e-cd6347d6bf69","Type":"ContainerDied","Data":"c84b4fd0a20e60157f5bf591e03eed9fcb3198a503e3a442355f70015b3b04d4"}
Nov 01 23:01:15.973589 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.973570    5057 scope.go:122] "RemoveContainer" containerID="b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7"
Nov 01 23:01:15.973879 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:01:15.973858    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7\": container with ID starting with b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7 not found: ID does not exist" containerID="b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7"
Nov 01 23:01:15.973978 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.973887    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7"} err="failed to get container status \"b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7\": rpc error: code = NotFound desc = could not find container \"b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7\": container with ID starting with b6cc28b028ff351364f65cdae70fe6b73a0c298a78a75d11aa792224a898d0d7 not found: ID does not exist"
Nov 01 23:01:15.973978 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.973904    5057 scope.go:122] "RemoveContainer" containerID="95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6"
Nov 01 23:01:15.982390 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.982373    5057 scope.go:122] "RemoveContainer" containerID="95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6"
Nov 01 23:01:15.982675 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:01:15.982656    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6\": container with ID starting with 95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6 not found: ID does not exist" containerID="95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6"
Nov 01 23:01:15.982738 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.982682    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6"} err="failed to get container status \"95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6\": rpc error: code = NotFound desc = could not find container \"95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6\": container with ID starting with 95463a2704944ef74c3c5734ffb1e166d4f82a4bfa969732b05ab08c5b5abaa6 not found: ID does not exist"
Nov 01 23:01:15.990724 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.990702    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"]
Nov 01 23:01:15.995459 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:15.995438    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-wl9g5"]
Nov 01 23:01:16.008631 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:16.008597    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"]
Nov 01 23:01:16.014642 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:16.014619    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-tpppl"]
Nov 01 23:01:16.898841 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:16.898801    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:01:17.396788 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:17.396752    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" path="/var/lib/kubelet/pods/58ff371d-d2ed-415c-988f-cffea5d08b83/volumes"
Nov 01 23:01:17.397386 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:17.397369    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" path="/var/lib/kubelet/pods/75a10a58-ea8f-437c-bb3e-cd6347d6bf69/volumes"
Nov 01 23:01:20.271710 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:20.271669    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:20.737666 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:20.737627    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:20.868826 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:20.868787    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:24.157845 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:24.157806    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:24.316656 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:24.316627    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:25.394900 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.394858    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:25.469072 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469036    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"]
Nov 01 23:01:25.469665 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469638    5057 cpu_manager.go:414] "containerMap: removing container" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main"
Nov 01 23:01:25.469665 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469663    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469690    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469698    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469714    5057 cpu_manager.go:414] "containerMap: removing container" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469722    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469734    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469741    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469755    5057 cpu_manager.go:414] "containerMap: removing container" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469762    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469773    5057 cpu_manager.go:414] "containerMap: removing container" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469781    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469800    5057 cpu_manager.go:414] "containerMap: removing container" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469807    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469824    5057 cpu_manager.go:414] "containerMap: removing container" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469832    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469846    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469853    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469868    5057 cpu_manager.go:414] "containerMap: removing container" podUID="5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" containerName="maas-api"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469876    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" containerName="maas-api"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469888    5057 cpu_manager.go:414] "containerMap: removing container" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469896    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469907    5057 cpu_manager.go:414] "containerMap: removing container" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469914    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469936    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a741756c-9713-4bad-981f-0f1b25ffa080" containerName="manager"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469942    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a741756c-9713-4bad-981f-0f1b25ffa080" containerName="manager"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469953    5057 cpu_manager.go:414] "containerMap: removing container" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469960    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469970    5057 cpu_manager.go:414] "containerMap: removing container" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469977    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469988    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.469994    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470006    5057 cpu_manager.go:414] "containerMap: removing container" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470013    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470028    5057 cpu_manager.go:414] "containerMap: removing container" podUID="619f00b4-a838-4c64-96e4-7e1fcb83d1fe" containerName="discovery"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470035    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="619f00b4-a838-4c64-96e4-7e1fcb83d1fe" containerName="discovery"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470047    5057 cpu_manager.go:414] "containerMap: removing container" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470054    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470065    5057 cpu_manager.go:414] "containerMap: removing container" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy"
Nov 01 23:01:25.470120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470073    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470174    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470188    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a741756c-9713-4bad-981f-0f1b25ffa080" containerName="manager"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470200    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="c82a6301-b1dc-4116-9f89-95dc2ccec308" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470210    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="eeaab9d5-3ace-4461-82a1-07e5b0437dac" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470221    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7c6d58e-2db3-4e3f-a2ae-3917cf8b52a3" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470230    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="61bbc377-4c9a-46b8-b8d7-05bbe87be152" containerName="main"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470239    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="619f00b4-a838-4c64-96e4-7e1fcb83d1fe" containerName="discovery"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470248    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1b24ec1-6153-4575-83b2-4f894618dd72" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470256    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="58ff371d-d2ed-415c-988f-cffea5d08b83" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470265    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="d2c13e54-c14f-46c5-bd5d-03414f2534b1" containerName="main"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470274    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1d8dc09-b7c3-434a-9595-d50e5cb06f77" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470284    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="6b4a3bd2-7201-44e5-9f11-0a0bb1b38cf1" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470292    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470301    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="75a10a58-ea8f-437c-bb3e-cd6347d6bf69" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470308    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="b5c05fa8-2874-4cb3-ba0a-f570a02800a6" containerName="main"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470320    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe9cbfb3-42ea-4b6e-a373-699f5daed5f6" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470329    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="94b34e89-e83e-406d-8c6b-8589cbd6fd63" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470338    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="a9cb7b00-e3a7-4264-a7c3-34b53ca35d84" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470348    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="5ef56ea1-0f1e-4553-8f34-b2a508dfcff8" containerName="maas-api"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470355    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="c22e4429-a16f-49c7-b7e0-4b2eefc359ad" containerName="istio-proxy"
Nov 01 23:01:25.472126 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.470363    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="afbfc2e3-7c03-40b2-a82b-be84c738f252" containerName="istio-proxy"
Nov 01 23:01:25.475537 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.475515    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.495616 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.495585    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"]
Nov 01 23:01:25.560350 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.560311    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:25.642078 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642026    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642233 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642115    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642233 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642167    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642233 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642198    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642380 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642232    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c13087-b188-46c3-a94c-68b8b8b81448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642380 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642265    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c13087-b188-46c3-a94c-68b8b8b81448-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642380 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642296    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w96g5\" (UniqueName: \"kubernetes.io/projected/73c13087-b188-46c3-a94c-68b8b8b81448-kube-api-access-w96g5\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642575 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642518    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.642575 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.642542    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c13087-b188-46c3-a94c-68b8b8b81448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.737460 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.737427    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:25.743559 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743522    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.743681 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743577    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.743737 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743715    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.743785 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743769    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.743834 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743818    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c13087-b188-46c3-a94c-68b8b8b81448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.743888 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743875    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c13087-b188-46c3-a94c-68b8b8b81448-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.743952 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743918    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w96g5\" (UniqueName: \"kubernetes.io/projected/73c13087-b188-46c3-a94c-68b8b8b81448-kube-api-access-w96g5\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744005 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743966    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744005 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.743990    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c13087-b188-46c3-a94c-68b8b8b81448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744116 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.744053    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744192 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.744158    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744254 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.744181    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744357 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.744335    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.744653 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.744630    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/73c13087-b188-46c3-a94c-68b8b8b81448-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.745994 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.745975    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/73c13087-b188-46c3-a94c-68b8b8b81448-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.746486 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.746465    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/73c13087-b188-46c3-a94c-68b8b8b81448-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.753424 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.753395    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w96g5\" (UniqueName: \"kubernetes.io/projected/73c13087-b188-46c3-a94c-68b8b8b81448-kube-api-access-w96g5\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.753536 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.753471    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73c13087-b188-46c3-a94c-68b8b8b81448-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-9zrmc\" (UID: \"73c13087-b188-46c3-a94c-68b8b8b81448\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:25.790543 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:25.790457    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:26.081504 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:26.081465    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"]
Nov 01 23:01:26.087630 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:26.087591    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:01:26.087762 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:26.087686    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:01:26.087762 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:26.087743    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:01:26.351536 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:26.351350    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:27.044299 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.043966    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" event={"ID":"73c13087-b188-46c3-a94c-68b8b8b81448","Type":"ContainerStarted","Data":"441e641eca7f2cd945ebd345312a3ead630160542d5cfd0072f19c1d3894d1fb"}
Nov 01 23:01:27.044299 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.044049    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" event={"ID":"73c13087-b188-46c3-a94c-68b8b8b81448","Type":"ContainerStarted","Data":"7b5a90cbe557f49ea0f91d018b9f5a1c61a27b17e73454880a9b2ca78d089d99"}
Nov 01 23:01:27.102397 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.102336    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podStartSLOduration=2.102272134 podStartE2EDuration="2.102272134s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 23:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 23:01:27.098881848 +0000 UTC m=+2343.840577491" watchObservedRunningTime="2026-11-01 23:01:27.102272134 +0000 UTC m=+2343.843967753"
Nov 01 23:01:27.363324 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.363193    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:27.790670 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.790629    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:27.792618 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.792584    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 23:01:27.792726 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:27.792643    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 23:01:28.225516 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:28.224631    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:01:28.265229 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:28.265180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:28.791097 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:28.791059    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 23:01:28.791260 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:28.791113    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 23:01:29.350520 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:29.349484    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:29.791638 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:29.791604    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 23:01:29.791828 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:29.791658    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 23:01:30.401459 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:30.401417    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:30.791048 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:30.790996    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 23:01:30.791271 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:30.791061    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 23:01:31.645514 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:31.640466    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:31.791583 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:31.791542    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 23:01:31.791806 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:31.791597    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 23:01:32.643075 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:32.643035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:32.791871 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:32.791825    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused" start-of-body=
Nov 01 23:01:32.792258 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:32.791901    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": dial tcp 10.132.0.115:15021: connect: connection refused"
Nov 01 23:01:33.542660 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:33.542617    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:34.519223 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:34.519191    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:34.519717 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:34.519529    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:34.520579 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:34.520554    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc"
Nov 01 23:01:34.611448 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:34.611393    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:35.578847 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:35.578811    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:36.389219 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:36.389183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:37.096160 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:37.096127    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:37.805785 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:37.805745    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:38.494764 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:38.494726    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:39.101880 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:39.101820    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:39.237118 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:39.237080    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:39.816052 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:39.816016    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:39.977597 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:39.977562    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:40.015481 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:01:40.015444    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:01:40.301910 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:40.301871    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:41.246543 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:41.246507    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:41.928344 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:41.928309    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:42.054634 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:42.054595    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:42.842423 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:42.842388    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:43.406919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:43.406883    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:01:43.729144 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:43.729100    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:44.615804 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:44.615768    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:45.310278 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:45.310247    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:46.010515 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:46.006945    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:46.692423 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:46.692382    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:47.414544 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:47.414485    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:48.101616 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:48.101579    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:48.824293 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:48.824259    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:49.521537 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:49.521484    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:50.406201 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:50.406165    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:51.212060 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:51.212023    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:51.887761 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:51.887727    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:52.581149 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:52.581117    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:53.303640 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:53.303605    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:53.999302 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:53.999263    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:54.814957 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:54.814910    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:55.518518 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:55.518442    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:56.305838 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:56.305797    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:57.047383 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:57.047341    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:57.787755 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:57.787716    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:58.615153 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:58.615116    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:01:59.422535 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:59.422488    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:01:59.615294 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:01:59.615251    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:00.311471 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:00.311434    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:01.012716 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:01.012681    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:01.639557 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:01.639506    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:02.507182 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:02.507144    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:03.191426 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:03.191382    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:03.939263 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:03.939227    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:04.608866 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:04.608831    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:05.315938 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:05.315902    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:06.016626 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:06.016579    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:06.697073 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:06.697035    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:07.517673 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:07.517643    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:08.102751 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:08.102694    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:08.830897 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:08.830849    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:09.525272 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:09.525238    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:10.014991 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:02:10.014957    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:02:10.239035 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:10.238986    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:10.908843 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:10.908807    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:11.619722 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:11.619688    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:12.302212 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:12.302175    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:13.105947 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:13.105912    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:13.797717 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:13.797685    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:14.424690 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:14.424644    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:14.591061 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:14.591026    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:15.401979 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:15.401938    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:16.007336 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:16.007286    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:16.701600 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:16.701564    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:17.429059 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:17.429022    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:18.152502 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:18.152451    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:18.923877 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:18.923838    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:19.591233 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:19.591196    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:20.288681 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:20.288636    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:20.993428 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:20.993376    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:21.789089 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:21.789053    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:21.906160 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:21.906118    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:22.592590 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:22.592527    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:22.602857 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:22.602817    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:02:23.312967 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:23.312910    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:23.789620 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:23.789595    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:02:23.801699 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:23.801667    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:02:24.002014 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:24.001964    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:24.716859 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:24.716823    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:25.404167 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:25.404128    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:26.099939 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:26.099901    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:26.789134 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:26.789094    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:26.894537 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:26.894488    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:27.631585 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:27.631546    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:28.395337 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:28.395307    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:29.192095 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:29.192056    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:29.886730 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:29.886690    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:30.599888 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:30.599847    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:31.285579 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:31.285532    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:32.006821 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:32.006786    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:32.333348 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:32.333257    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:32.506634 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:32.506595    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:33.040673 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:33.040639    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:33.719662 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:33.719625    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:33.890447 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:33.890409    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:34.015319 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:34.015216    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:34.513864 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:34.513830    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:35.381609 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:35.381570    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:02:40.014132 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:02:40.014092    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:02:40.506433 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:40.506396    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:40.698554 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:40.698488    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:40.995171 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:40.995136    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:44.308251 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:44.308212    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:44.568633 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:44.568552    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:50.171896 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:50.171857    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:02:53.615112 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:53.615078    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:02:58.025624 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:02:58.025586    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:03:00.656266 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:00.656227    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"]
Nov 01 23:03:00.656757 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:00.656432    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" podUID="5c6901d3-05eb-4d28-8543-76ec9d234180" containerName="manager" containerID="cri-o://0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da" gracePeriod=10
Nov 01 23:03:00.692770 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:00.692714    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" podUID="5c6901d3-05eb-4d28-8543-76ec9d234180" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.37:8081/readyz\": dial tcp 10.132.0.37:8081: connect: connection refused"
Nov 01 23:03:00.922916 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:00.921623    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 23:03:01.066182 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.066138    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume\") pod \"5c6901d3-05eb-4d28-8543-76ec9d234180\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") "
Nov 01 23:03:01.066392 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.066194    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz\" (UniqueName: \"kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz\") pod \"5c6901d3-05eb-4d28-8543-76ec9d234180\" (UID: \"5c6901d3-05eb-4d28-8543-76ec9d234180\") "
Nov 01 23:03:01.066675 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.066643    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume" pod "5c6901d3-05eb-4d28-8543-76ec9d234180" (UID: "5c6901d3-05eb-4d28-8543-76ec9d234180"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Nov 01 23:03:01.068392 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.068370    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz" pod "5c6901d3-05eb-4d28-8543-76ec9d234180" (UID: "5c6901d3-05eb-4d28-8543-76ec9d234180"). InnerVolumeSpecName "kube-api-access-hf8jz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:03:01.167936 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.167895    5057 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c6901d3-05eb-4d28-8543-76ec9d234180-extensions-socket-volume\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:03:01.167936 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.167929    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hf8jz\" (UniqueName: \"kubernetes.io/projected/5c6901d3-05eb-4d28-8543-76ec9d234180-kube-api-access-hf8jz\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:03:01.528912 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.528873    5057 generic.go:356] "Generic (PLEG): container finished" podID="5c6901d3-05eb-4d28-8543-76ec9d234180" containerID="0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da" exitCode=0
Nov 01 23:03:01.529127 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.528924    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"
Nov 01 23:03:01.529127 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.528944    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" event={"ID":"5c6901d3-05eb-4d28-8543-76ec9d234180","Type":"ContainerDied","Data":"0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da"}
Nov 01 23:03:01.529127 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.529004    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv" event={"ID":"5c6901d3-05eb-4d28-8543-76ec9d234180","Type":"ContainerDied","Data":"c341cc9f6f5427a2f74673a7228f04395c51f621f55e333436f28e9a8a1fdf94"}
Nov 01 23:03:01.529127 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.529029    5057 scope.go:122] "RemoveContainer" containerID="0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da"
Nov 01 23:03:01.540594 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.540574    5057 scope.go:122] "RemoveContainer" containerID="0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da"
Nov 01 23:03:01.540937 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:03:01.540908    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da\": container with ID starting with 0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da not found: ID does not exist" containerID="0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da"
Nov 01 23:03:01.541016 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.540946    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da"} err="failed to get container status \"0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da\": rpc error: code = NotFound desc = could not find container \"0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da\": container with ID starting with 0af00b5982ff42b7f9d1815c5504ce54b495edf921e460c929ab09a3a0d3f3da not found: ID does not exist"
Nov 01 23:03:01.546090 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.546061    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"]
Nov 01 23:03:01.554175 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:01.554144    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-dtjcv"]
Nov 01 23:03:03.397335 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:03.397300    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c6901d3-05eb-4d28-8543-76ec9d234180" path="/var/lib/kubelet/pods/5c6901d3-05eb-4d28-8543-76ec9d234180/volumes"
Nov 01 23:03:10.016279 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:03:10.016234    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:03:10.596505 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.596459    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"]
Nov 01 23:03:10.597066 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.597038    5057 cpu_manager.go:414] "containerMap: removing container" podUID="5c6901d3-05eb-4d28-8543-76ec9d234180" containerName="manager"
Nov 01 23:03:10.597066 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.597061    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c6901d3-05eb-4d28-8543-76ec9d234180" containerName="manager"
Nov 01 23:03:10.597281 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.597081    5057 cpu_manager.go:414] "containerMap: removing container" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy"
Nov 01 23:03:10.597281 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.597090    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a18890f8-c3a4-4b2b-a261-983211e39f49" containerName="istio-proxy"
Nov 01 23:03:10.597281 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.597165    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c6901d3-05eb-4d28-8543-76ec9d234180" containerName="manager"
Nov 01 23:03:10.602041 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.602019    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.615643 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.615618    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"]
Nov 01 23:03:10.758395 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758353    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758644 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758415    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758644 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758546    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758644 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758568    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758644 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758595    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758644 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758620    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wxqc\" (UniqueName: \"kubernetes.io/projected/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-kube-api-access-8wxqc\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758903 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758743    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758903 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758791    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.758973 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.758891    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.859577 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859469    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.859577 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859544    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.859788 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859580    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.859788 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859606    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wxqc\" (UniqueName: \"kubernetes.io/projected/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-kube-api-access-8wxqc\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.859788 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859641    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860047 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859773    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860047 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859829    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860047 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859905    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860047 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.859954    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860275 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.860093    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-workload-certs\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860275 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.860152    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-workload-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860275 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.860265    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-data\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860433 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.860344    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.860433 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.860346    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-credential-socket\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.862112 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.862090    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.862204 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.862115    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-envoy\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.868983 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.868957    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wxqc\" (UniqueName: \"kubernetes.io/projected/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-kube-api-access-8wxqc\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.869076 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.869009    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a077e74-c5a6-4e89-8ff3-94ac4df0188e-istio-token\") pod \"maas-default-gateway-openshift-default-58847f9fc5-kbn9c\" (UID: \"9a077e74-c5a6-4e89-8ff3-94ac4df0188e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:10.917004 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:10.916971    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:11.195583 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.195554    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"]
Nov 01 23:03:11.198002 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:03:11.197972    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a077e74_c5a6_4e89_8ff3_94ac4df0188e.slice/crio-4ca593384176d25f183e3acd703c93054401e5a1c3c6a06a4755c3bedcd8f165 WatchSource:0}: Error finding container 4ca593384176d25f183e3acd703c93054401e5a1c3c6a06a4755c3bedcd8f165: Status 404 returned error can't find the container with id 4ca593384176d25f183e3acd703c93054401e5a1c3c6a06a4755c3bedcd8f165
Nov 01 23:03:11.200131 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.200098    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:03:11.200215 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.200161    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:03:11.200215 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.200206    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:03:11.580732 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.580693    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" event={"ID":"9a077e74-c5a6-4e89-8ff3-94ac4df0188e","Type":"ContainerStarted","Data":"47ba9f84b384d8c977062b01e23a7ea13caa8eb420ad730a615e3fb242f44b4c"}
Nov 01 23:03:11.580925 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.580757    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" event={"ID":"9a077e74-c5a6-4e89-8ff3-94ac4df0188e","Type":"ContainerStarted","Data":"4ca593384176d25f183e3acd703c93054401e5a1c3c6a06a4755c3bedcd8f165"}
Nov 01 23:03:11.608814 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.608758    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podStartSLOduration=1.6087418009999999 podStartE2EDuration="1.608741801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 23:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 23:03:11.607488518 +0000 UTC m=+2448.349184155" watchObservedRunningTime="2026-11-01 23:03:11.608741801 +0000 UTC m=+2448.350437419"
Nov 01 23:03:11.918113 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:11.918029    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:12.923675 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:12.923646    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:13.592225 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:13.592196    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:13.593306 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:13.593285    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:03:14.468607 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:14.468572    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:03:15.140434 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:15.140392    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:03:15.329898 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:03:15.329861    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:03:40.014453 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:03:40.014367    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:04:09.268348 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.268310    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"]
Nov 01 23:04:09.272060 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.272040    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.275455 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.275429    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gptd7\""
Nov 01 23:04:09.283699 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.283672    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"]
Nov 01 23:04:09.319058 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.319012    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0096d7f-77fb-406d-ab0d-6ee3f70f007f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-5h446\" (UID: \"a0096d7f-77fb-406d-ab0d-6ee3f70f007f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.319251 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.319067    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cznnd\" (UniqueName: \"kubernetes.io/projected/a0096d7f-77fb-406d-ab0d-6ee3f70f007f-kube-api-access-cznnd\") pod \"kuadrant-operator-controller-manager-678d87766f-5h446\" (UID: \"a0096d7f-77fb-406d-ab0d-6ee3f70f007f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.420025 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.419985    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0096d7f-77fb-406d-ab0d-6ee3f70f007f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-5h446\" (UID: \"a0096d7f-77fb-406d-ab0d-6ee3f70f007f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.420025 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.420031    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cznnd\" (UniqueName: \"kubernetes.io/projected/a0096d7f-77fb-406d-ab0d-6ee3f70f007f-kube-api-access-cznnd\") pod \"kuadrant-operator-controller-manager-678d87766f-5h446\" (UID: \"a0096d7f-77fb-406d-ab0d-6ee3f70f007f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.420396 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.420374    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a0096d7f-77fb-406d-ab0d-6ee3f70f007f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-678d87766f-5h446\" (UID: \"a0096d7f-77fb-406d-ab0d-6ee3f70f007f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.429615 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.429577    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cznnd\" (UniqueName: \"kubernetes.io/projected/a0096d7f-77fb-406d-ab0d-6ee3f70f007f-kube-api-access-cznnd\") pod \"kuadrant-operator-controller-manager-678d87766f-5h446\" (UID: \"a0096d7f-77fb-406d-ab0d-6ee3f70f007f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.586611 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.586522    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:09.868705 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:09.868679    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"]
Nov 01 23:04:09.870391 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:04:09.870368    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0096d7f_77fb_406d_ab0d_6ee3f70f007f.slice/crio-cd63215b75bc1018e572a9b25fa469a88b143d23d0b40daae3fe002bc528eb87 WatchSource:0}: Error finding container cd63215b75bc1018e572a9b25fa469a88b143d23d0b40daae3fe002bc528eb87: Status 404 returned error can't find the container with id cd63215b75bc1018e572a9b25fa469a88b143d23d0b40daae3fe002bc528eb87
Nov 01 23:04:10.015209 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:04:10.015170    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:04:10.867136 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:10.867099    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446" event={"ID":"a0096d7f-77fb-406d-ab0d-6ee3f70f007f","Type":"ContainerStarted","Data":"23e5198715c4bf835f7ac22fb2ad4f71b2f6644c353e0571ed66c5e08d6c03c0"}
Nov 01 23:04:10.867629 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:10.867157    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446" event={"ID":"a0096d7f-77fb-406d-ab0d-6ee3f70f007f","Type":"ContainerStarted","Data":"cd63215b75bc1018e572a9b25fa469a88b143d23d0b40daae3fe002bc528eb87"}
Nov 01 23:04:10.867629 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:10.867240    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:10.888761 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:10.888704    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446" podStartSLOduration=1.888687085 podStartE2EDuration="1.888687085s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 23:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 23:04:10.886737063 +0000 UTC m=+2507.628432681" watchObservedRunningTime="2026-11-01 23:04:10.888687085 +0000 UTC m=+2507.630382700"
Nov 01 23:04:14.505329 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:14.505285    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:04:21.875613 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:21.875582    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5h446"
Nov 01 23:04:30.004975 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:30.004937    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:04:32.487705 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:32.487617    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:04:32.487705 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:32.487691    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:04:40.013324 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:04:40.013286    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:04:40.031676 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:40.031648    5057 generic.go:356] "Generic (PLEG): container finished" podID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerID="47ba9f84b384d8c977062b01e23a7ea13caa8eb420ad730a615e3fb242f44b4c" exitCode=0
Nov 01 23:04:40.031814 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:40.031715    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" event={"ID":"9a077e74-c5a6-4e89-8ff3-94ac4df0188e","Type":"ContainerDied","Data":"47ba9f84b384d8c977062b01e23a7ea13caa8eb420ad730a615e3fb242f44b4c"}
Nov 01 23:04:40.032096 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:40.032083    5057 scope.go:122] "RemoveContainer" containerID="47ba9f84b384d8c977062b01e23a7ea13caa8eb420ad730a615e3fb242f44b4c"
Nov 01 23:04:40.033251 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:40.033225    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:04:40.033311 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:40.033297    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:04:40.033352 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:40.033334    5057 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236064Ki","pods":"250"}
Nov 01 23:04:41.045372 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:41.045337    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" event={"ID":"9a077e74-c5a6-4e89-8ff3-94ac4df0188e","Type":"ContainerStarted","Data":"aa733c9e87718be398cc65e1569d286ce2902711813d6ee10ee44956008837a9"}
Nov 01 23:04:41.927517 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:41.918425    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:04:41.927517 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:41.923584    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 23:04:41.927517 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:41.923637    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 23:04:42.926521 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:42.919932    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 23:04:42.926521 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:42.919989    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 23:04:43.592830 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:43.592796    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:04:43.917448 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:43.917359    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 23:04:43.917448 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:43.917414    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 23:04:44.921892 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:44.921854    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 23:04:44.922529 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:44.922470    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 23:04:45.237518 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:45.236418    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:04:45.921520 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:45.917819    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 23:04:45.921520 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:45.917889    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 23:04:46.917574 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:46.917530    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused" start-of-body=
Nov 01 23:04:46.918001 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:46.917601    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": dial tcp 10.132.0.116:15021: connect: connection refused"
Nov 01 23:04:48.925202 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:48.924584    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:04:48.925202 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:48.924652    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:04:49.261514 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:49.260564    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:04:49.935638 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:49.935591    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:04:49.936246 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:49.936210    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:04:50.977715 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:50.977669    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:04:50.978225 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:50.977749    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:04:51.978904 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:51.978863    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:04:51.979368 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:51.978935    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:04:53.020905 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:53.020790    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-kbn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:04:53.020905 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:53.020871    5057 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c" podUID="9a077e74-c5a6-4e89-8ff3-94ac4df0188e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.116:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:04:53.619576 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:53.619544    5057 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:04:53.620821 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:04:53.620794    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-kbn9c"
Nov 01 23:05:06.017644 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:06.017603    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:05:06.837881 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:06.837834    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:07.404515 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:07.396323    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:07.648943 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:07.648885    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:10.013121 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:05:10.013080    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:05:10.446200 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:10.446097    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:10.591508 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:10.591456    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:11.661660 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:11.661623    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:11.826215 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:11.826183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:11.945681 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:11.945593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:12.616918 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:12.616879    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:13.500039 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:13.500003    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:13.801855 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:13.801773    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:05:14.330220 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:14.330181    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:14.562067 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:14.562030    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:15.356480 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:15.356448    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:15.985713 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:15.985671    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:16.743463 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:16.743428    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:17.393216 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:17.393177    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:18.122484 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:18.122441    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:18.801100 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:18.801058    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:19.537826 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:19.537777    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:20.443270 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:20.443230    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:21.098379 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:21.098346    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:21.784280 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:21.784238    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:22.494416 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:22.494372    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:23.330542 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:23.330479    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:23.991097 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:23.991065    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:24.757589 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:24.757551    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:25.394132 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:25.394094    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:25.618803 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:25.618768    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:26.047819 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:26.047786    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:26.219902 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:26.219863    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:26.505773 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:26.505737    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:27.247602 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:27.247558    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:27.610353 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:27.610252    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:28.627548 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:28.627471    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:28.985989 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:28.985951    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:05:29.434293 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:29.434205    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:29.584221 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:29.584187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:30.310985 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:30.310940    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:31.007230 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:31.007195    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:31.713429 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:31.713390    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:32.411523 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:32.411459    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:33.264651 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:33.264619    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:33.887067 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:33.887026    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:34.600040 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:34.600009    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:35.280531 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:35.280484    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:35.952123 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:35.952087    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:36.630168 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:36.630129    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:37.297580 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:37.297542    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:37.987722 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:37.987685    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:38.699989 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:38.699956    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:39.424590 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:39.424558    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:40.012981 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:05:40.012940    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:05:40.219825 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:40.219788    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:40.899573 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:40.899528    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:41.692637 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:41.692600    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:42.397689 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:42.397654    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:43.118554 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:43.118520    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:43.964521 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:43.964473    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:44.689363 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:44.689320    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:45.502875 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:45.502837    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:45.698598 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:45.698562    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:05:46.216704 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:46.216664    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:46.898622 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:46.898584    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:47.693175 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:47.693143    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:48.426832 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:48.426789    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:49.111525 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:49.111471    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:49.904306 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:49.904271    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:50.616623 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:50.616585    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:51.306104 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:51.306061    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:52.042154 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:52.042121    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:52.510634 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:52.510590    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:52.762302 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:52.762205    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:53.411319 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:53.411285    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:54.083259 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:54.083228    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:54.786549 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:54.786517    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:55.464794 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:55.464758    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:56.174980 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:56.174943    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:56.914026 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:56.913991    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:57.879125 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:57.879079    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:58.609432 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:58.609382    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:59.287990 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:59.287950    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:05:59.990803 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:05:59.990766    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:00.696401 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:00.696355    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:01.493341 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:01.493308    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:02.301360 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:02.301327    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:02.995779 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:02.995744    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:03.680009 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:03.679967    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:04.493183 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:04.493143    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:05.165761 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:05.165725    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:05.792998 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:05.792959    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:06.622371 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:06.622335    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:07.263869 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:07.263821    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:07.906053 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:07.906008    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:08.114297 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:08.114261    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:08.404783 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:08.404746    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:06:08.869880 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:08.869839    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:09.582724 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:09.582688    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:10.016257 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:06:10.016214    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:06:10.308046 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:10.307958    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:11.026133 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:11.026037    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:11.713416 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:11.713372    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:12.381317 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:12.381277    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:13.174455 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:13.174420    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:13.308253 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:13.308216    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:13.988631 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:13.988590    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:14.597428 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:14.597380    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:14.742060 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:14.742019    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:15.407903 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:15.407841    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:16.090722 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:16.090682    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:16.783674 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:16.783634    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:17.496269 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:17.496230    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:18.252137 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:18.252090    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:18.446896 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:18.446862    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:18.645943 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:18.645865    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:18.932288 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:18.932189    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:19.148707 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:19.148669    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:19.291741 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:19.291704    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:19.753389 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:19.753351    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:20.720444 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:20.720408    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:06:25.845713 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:25.845680    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:25.999985 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:25.999948    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:26.162576 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:26.162449    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:29.705949 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:29.705913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:31.119663 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:31.119621    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:06:39.251264 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:39.251164    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:39.370588 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:39.370555    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:40.014457 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:06:40.014415    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:06:46.900662 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:46.900621    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:48.518324 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:48.518283    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:06:50.597522 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:50.588582    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:50.836216 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:50.836183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:06:56.756742 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:06:56.756703    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:04.113964 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:04.113921    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:07:10.014033 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:07:10.013992    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:07:14.537306 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:14.537262    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:14.658735 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:14.658700    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:15.889550 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:15.889513    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:07:16.193843 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:16.193756    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:21.605085 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:21.605048    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:07:23.818651 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:23.818621    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:07:23.834661 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:23.834637    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:07:30.419838 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:30.419796    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:07:31.785276 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:31.785239    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:31.908341 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:31.908296    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:32.506901 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:32.506858    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-d6k7r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:07:32.507136 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:32.506951    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-d6k7r" podUID="a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:07:34.764700 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:34.764663    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:07:35.569518 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:35.568595    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:07:35.569518 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:35.568669    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:07:40.013423 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:07:40.013382    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:07:50.685083 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:50.685040    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:07:58.375261 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:58.375221    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:58.794728 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:58.794689    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:07:58.942338 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:07:58.942295    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:01.360297 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:01.360257    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:01.519536 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:01.519482    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:02.517980 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:02.517930    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:02.674878 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:02.674843    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:02.807959 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:02.807868    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:03.703877 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:03.703842    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:04.525737 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:04.525697    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:05.248140 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:05.248101    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:05.313460 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:05.313423    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:08:05.992891 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:05.992858    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:06.703505 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:06.703449    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:06.776192 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:06.776153    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"]
Nov 01 23:08:06.776418 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:06.776394    5057 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" podUID="d57b4deb-05d4-489f-add3-e43743fc7ece" containerName="maas-api" containerID="cri-o://7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1" gracePeriod=30
Nov 01 23:08:07.256360 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:07.256315    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" podUID="d57b4deb-05d4-489f-add3-e43743fc7ece" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.45:8443/health\": dial tcp 10.132.0.45:8443: connect: connection refused"
Nov 01 23:08:07.441475 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:07.441437    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:08.038106 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.038078    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 23:08:08.124272 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.124183    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:08.197730 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.197699    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca\") pod \"d57b4deb-05d4-489f-add3-e43743fc7ece\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") "
Nov 01 23:08:08.197730 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.197742    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl\" (UniqueName: \"kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl\") pod \"d57b4deb-05d4-489f-add3-e43743fc7ece\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") "
Nov 01 23:08:08.197940 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.197802    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs\") pod \"d57b4deb-05d4-489f-add3-e43743fc7ece\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") "
Nov 01 23:08:08.197940 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.197823    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls\") pod \"d57b4deb-05d4-489f-add3-e43743fc7ece\" (UID: \"d57b4deb-05d4-489f-add3-e43743fc7ece\") "
Nov 01 23:08:08.198154 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.198127    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca" pod "d57b4deb-05d4-489f-add3-e43743fc7ece" (UID: "d57b4deb-05d4-489f-add3-e43743fc7ece"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:08:08.200130 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.200098    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl" pod "d57b4deb-05d4-489f-add3-e43743fc7ece" (UID: "d57b4deb-05d4-489f-add3-e43743fc7ece"). InnerVolumeSpecName "kube-api-access-qlvnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:08:08.200238 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.200104    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls" pod "d57b4deb-05d4-489f-add3-e43743fc7ece" (UID: "d57b4deb-05d4-489f-add3-e43743fc7ece"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:08:08.200238 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.200135    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs" pod "d57b4deb-05d4-489f-add3-e43743fc7ece" (UID: "d57b4deb-05d4-489f-add3-e43743fc7ece"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:08:08.234042 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.234007    5057 generic.go:356] "Generic (PLEG): container finished" podID="d57b4deb-05d4-489f-add3-e43743fc7ece" containerID="7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1" exitCode=0
Nov 01 23:08:08.234187 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.234051    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"
Nov 01 23:08:08.234187 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.234097    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" event={"ID":"d57b4deb-05d4-489f-add3-e43743fc7ece","Type":"ContainerDied","Data":"7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1"}
Nov 01 23:08:08.234187 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.234155    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz" event={"ID":"d57b4deb-05d4-489f-add3-e43743fc7ece","Type":"ContainerDied","Data":"d4c10ec85d10a2429eb9b47af3fc88d0e14f7da759046c326159c7fe3bbf9cb8"}
Nov 01 23:08:08.234187 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.234178    5057 scope.go:122] "RemoveContainer" containerID="7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1"
Nov 01 23:08:08.244028 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.244010    5057 scope.go:122] "RemoveContainer" containerID="7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1"
Nov 01 23:08:08.244281 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:08:08.244266    5057 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1\": container with ID starting with 7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1 not found: ID does not exist" containerID="7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1"
Nov 01 23:08:08.244333 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.244290    5057 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1"} err="failed to get container status \"7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1\": rpc error: code = NotFound desc = could not find container \"7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1\": container with ID starting with 7f024d768c5ab243d18d6cdb21aa054c60cbb552812fb5c20dc5a91107debcd1 not found: ID does not exist"
Nov 01 23:08:08.259522 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.259480    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"]
Nov 01 23:08:08.263302 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.263274    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-54c8494dc4-c6skz"]
Nov 01 23:08:08.299003 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.298973    5057 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-metrics-certs\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:08.299003 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.298998    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d57b4deb-05d4-489f-add3-e43743fc7ece-maas-api-tls\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:08.299177 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.299019    5057 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d57b4deb-05d4-489f-add3-e43743fc7ece-openshift-service-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:08.299177 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.299029    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qlvnl\" (UniqueName: \"kubernetes.io/projected/d57b4deb-05d4-489f-add3-e43743fc7ece-kube-api-access-qlvnl\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:08.892852 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:08.892814    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:09.396165 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:09.396130    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d57b4deb-05d4-489f-add3-e43743fc7ece" path="/var/lib/kubelet/pods/d57b4deb-05d4-489f-add3-e43743fc7ece/volumes"
Nov 01 23:08:09.589464 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:09.589418    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:10.013346 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:08:10.013306    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:08:10.294805 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:10.294686    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:11.099085 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:11.099047    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:11.903831 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:11.903792    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:12.694065 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:12.694027    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:13.405692 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:13.405646    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:14.223033 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:14.222974    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:14.704632 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:14.704593    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:14.918780 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:14.918734    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:15.606198 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:15.606165    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:16.292302 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.292267    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:16.971310 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.971273    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"]
Nov 01 23:08:16.971732 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.971675    5057 cpu_manager.go:414] "containerMap: removing container" podUID="d57b4deb-05d4-489f-add3-e43743fc7ece" containerName="maas-api"
Nov 01 23:08:16.971732 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.971687    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d57b4deb-05d4-489f-add3-e43743fc7ece" containerName="maas-api"
Nov 01 23:08:16.971815 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.971778    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="d57b4deb-05d4-489f-add3-e43743fc7ece" containerName="maas-api"
Nov 01 23:08:16.976055 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.976034    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:16.978594 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.978569    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9gv8\" (UniqueName: \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:16.978731 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.978692    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:16.978786 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.978747    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:16.981714 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.981683    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 23:08:16.981830 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.981802    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-xlj2w\""
Nov 01 23:08:16.985252 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:16.983326    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"]
Nov 01 23:08:17.080020 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.079973    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.080020 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.080023    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9gv8\" (UniqueName: \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.080294 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.080108    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.080733 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.080705    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.082461 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.082435    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.088075 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:08:17.088057    5057 projected.go:196] Error preparing data for projected volume kube-api-access-n9gv8 for pod odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm: failed to fetch token: serviceaccounts "maas-api-cleanup" not found
Nov 01 23:08:17.088150 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:08:17.088118    5057 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8 podName:afc8d5a4-4da5-48a3-a064-82ab972a3fb6 nodeName:}" failed. No retries permitted until 2026-11-01 23:08:17.588099973 +0000 UTC m=+2754.329795570 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-n9gv8" (UniqueName: "kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8") pod "maas-api-revoke-keys-e2e-ait-75893881-sc6pm" (UID: "afc8d5a4-4da5-48a3-a064-82ab972a3fb6") : failed to fetch token: serviceaccounts "maas-api-cleanup" not found
Nov 01 23:08:17.261521 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.261416    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:17.685812 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.685720    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9gv8\" (UniqueName: \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.694652 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.694627    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9gv8\" (UniqueName: \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\") pod \"maas-api-revoke-keys-e2e-ait-75893881-sc6pm\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.890803 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.890764    5057 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm" secret="" err="secret \"maas-api-cleanup-dockercfg-xlj2w\" not found"
Nov 01 23:08:17.890964 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.890824    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:17.975852 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:17.975815    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:18.166923 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:18.166890    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"]
Nov 01 23:08:18.170238 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:08:18.170209    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc8d5a4_4da5_48a3_a064_82ab972a3fb6.slice/crio-b3072b00a6a75db386abdac468942cdeaf97dc0cf2eb0cea090fb75d42856046 WatchSource:0}: Error finding container b3072b00a6a75db386abdac468942cdeaf97dc0cf2eb0cea090fb75d42856046: Status 404 returned error can't find the container with id b3072b00a6a75db386abdac468942cdeaf97dc0cf2eb0cea090fb75d42856046
Nov 01 23:08:18.283597 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:18.283562    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm" event={"ID":"afc8d5a4-4da5-48a3-a064-82ab972a3fb6","Type":"ContainerStarted","Data":"b3072b00a6a75db386abdac468942cdeaf97dc0cf2eb0cea090fb75d42856046"}
Nov 01 23:08:18.707226 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:18.707189    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:19.290625 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:19.290533    5057 generic.go:356] "Generic (PLEG): container finished" podID="afc8d5a4-4da5-48a3-a064-82ab972a3fb6" containerID="f21d62ed79163700ce6ebd4e121adea2234e75338eecc951295f759a886944d9" exitCode=0
Nov 01 23:08:19.291009 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:19.290620    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm" event={"ID":"afc8d5a4-4da5-48a3-a064-82ab972a3fb6","Type":"ContainerDied","Data":"f21d62ed79163700ce6ebd4e121adea2234e75338eecc951295f759a886944d9"}
Nov 01 23:08:19.568317 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:19.568224    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:20.190835 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.190789    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:20.449294 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.449230    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:20.512115 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.512081    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\" (UniqueName: \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\") pod \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") "
Nov 01 23:08:20.512286 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.512219    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca\") pod \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") "
Nov 01 23:08:20.512286 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.512265    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca\") pod \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\" (UID: \"afc8d5a4-4da5-48a3-a064-82ab972a3fb6\") "
Nov 01 23:08:20.512679 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.512651    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca" pod "afc8d5a4-4da5-48a3-a064-82ab972a3fb6" (UID: "afc8d5a4-4da5-48a3-a064-82ab972a3fb6"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:08:20.514301 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.514268    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8" pod "afc8d5a4-4da5-48a3-a064-82ab972a3fb6" (UID: "afc8d5a4-4da5-48a3-a064-82ab972a3fb6"). InnerVolumeSpecName "kube-api-access-n9gv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:08:20.514507 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.514475    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca" pod "afc8d5a4-4da5-48a3-a064-82ab972a3fb6" (UID: "afc8d5a4-4da5-48a3-a064-82ab972a3fb6"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:08:20.613802 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.613767    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-openshift-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:20.613802 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.613796    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9gv8\" (UniqueName: \"kubernetes.io/projected/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-kube-api-access-n9gv8\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:20.614021 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.613829    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/afc8d5a4-4da5-48a3-a064-82ab972a3fb6-maas-api-xks-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:08:20.881155 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:20.881123    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:21.309776 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:21.309747    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"
Nov 01 23:08:21.309991 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:21.309745    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm" event={"ID":"afc8d5a4-4da5-48a3-a064-82ab972a3fb6","Type":"ContainerDied","Data":"b3072b00a6a75db386abdac468942cdeaf97dc0cf2eb0cea090fb75d42856046"}
Nov 01 23:08:21.309991 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:21.309916    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b3072b00a6a75db386abdac468942cdeaf97dc0cf2eb0cea090fb75d42856046"
Nov 01 23:08:21.557251 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:21.557216    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:22.198962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:22.198923    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:22.888702 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:22.888669    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:23.625256 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:23.625215    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:24.288155 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:24.288115    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:25.081982 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:25.081946    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:25.704757 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:25.704707    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:26.486167 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:26.486119    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:27.228354 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:27.228317    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:27.363664 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:27.363629    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:27.949107 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:27.949066    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:28.107284 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:28.107246    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"]
Nov 01 23:08:28.109343 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:28.109313    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-75893881-sc6pm"]
Nov 01 23:08:28.118759 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:28.118713    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:28.246172 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:28.246131    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:29.326268 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:29.326229    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:08:29.400528 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:29.400476    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="afc8d5a4-4da5-48a3-a064-82ab972a3fb6" path="/var/lib/kubelet/pods/afc8d5a4-4da5-48a3-a064-82ab972a3fb6/volumes"
Nov 01 23:08:33.184888 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:33.184827    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:34.349535 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:34.349484    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:34.751273 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:34.751228    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:35.038872 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:35.038778    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:37.663997 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:37.663964    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:37.800964 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:37.800934    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:38.879679 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:38.879641    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:39.051939 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:39.051895    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:39.194546 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:39.194414    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:39.711327 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:39.711280    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:40.015749 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:08:40.015661    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:08:40.489303 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:40.489258    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:41.117455 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:41.117419    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:41.435158 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:41.435070    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:08:42.053015 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:42.052975    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:42.826676 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:42.826627    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:43.517939 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:43.517885    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:44.201823 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:44.201783    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:44.911877 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:44.911835    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:45.632217 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:45.632180    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:46.472962 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:46.472922    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:47.189038 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:47.188997    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:47.988187 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:47.988148    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:48.787521 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:48.787459    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:49.578549 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:49.578509    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:50.256926 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:50.256890    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:51.189941 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:51.189907    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:52.049596 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:52.049556    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:52.631884 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:52.631849    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:52.881445 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:52.881390    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:53.656058 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:53.655999    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:54.421102 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:54.421062    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:54.548234 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:54.548156    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:55.413889 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:55.413849    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:56.164090 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:56.164053    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:56.927204 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:56.927126    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:57.711331 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:57.711284    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:58.446605 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:58.446567    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:59.227572 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:59.227534    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:08:59.925964 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:08:59.925923    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:00.594243 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:00.594202    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:01.294805 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:01.294768    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:01.906570 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:01.906533    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:02.597011 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:02.596977    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:03.402642 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:03.402603    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:04.101299 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:04.101263    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:04.792621 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:04.792574    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:05.495736 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:05.495698    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:06.129821 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:06.129783    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:06.790949 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:06.790913    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:07.501434 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:07.501401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:08.174225 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:08.174188    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:08.976014 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:08.975970    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:09.586348 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:09.586306    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:10.014782 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:09:10.014745    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:09:10.501633 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:10.501598    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:11.199485 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:11.199446    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:11.895774 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:11.895740    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:12.823671 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:12.823632    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:13.135759 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:13.135669    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:13.665462 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:13.665422    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:14.395469 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:14.395429    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:14.732054 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:14.732010    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:15.233657 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:15.233619    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:15.893929 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:15.893891    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:16.617858 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:16.617823    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:17.408016 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:17.407903    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:19.572451 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:19.572413    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:20.410254 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:20.410210    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:21.099931 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:21.099891    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:21.843701 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:21.843668    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:22.499224 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:22.499187    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:23.310249 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:23.310207    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:24.112708 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:24.112660    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:24.795681 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:24.795640    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:25.503610 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:25.503573    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:26.103424 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:26.103382    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:26.818510 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:26.818458    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:27.587845 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:27.587809    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:28.301305 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:28.301265    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:28.996680 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:28.996631    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:29.694095 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:29.694052    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:30.371587 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:30.371552    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:31.025196 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:31.025157    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:31.884655 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:31.884620    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:32.496155 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:32.496119    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:33.200280 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:33.200245    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:33.892989 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:33.892936    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:34.572093 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:34.572054    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:35.203396 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:35.203357    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:35.873055 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:35.873020    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:36.603628 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:36.603401    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:37.401712 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:37.401671    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:38.088259 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:38.088217    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:38.783035 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:38.782983    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:39.473033 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:39.472999    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:40.014941 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:09:40.014901    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:09:40.197001 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:40.196966    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:40.884549 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:40.884481    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:41.608215 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:41.608178    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:42.293223 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:42.293186    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:42.998028 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:42.997989    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:43.696939 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:43.696899    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:44.407336 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:44.407297    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:45.093181 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:45.093139    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:45.906105 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:45.906071    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:46.581349 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:46.581313    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:47.524886 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:47.524830    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:48.199313 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:48.199273    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:48.896527 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:48.896469    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:49.594110 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:49.594071    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:50.295910 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:50.295876    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:50.998628 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:50.998591    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:51.689503 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:51.689451    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:52.410098 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:52.410061    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:53.203220 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:53.203188    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:53.888633 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:53.888600    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:54.646059 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:54.646015    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:55.415474 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:55.415430    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:56.207820 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:56.207783    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:57.005836 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:57.005800    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:57.696048 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:57.696007    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:58.391648 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:58.391603    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:59.212260 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:59.212222    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:09:59.910820 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:09:59.910784    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:00.708173 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:00.708134    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:01.510671 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:01.510630    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:02.214510 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:02.214466    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:02.907408 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:02.907321    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:03.606207 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:03.606172    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:04.319271 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:04.319233    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:05.080364 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:05.080328    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:05.782818 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:05.782776    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:06.588334 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:06.588299    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:07.290530 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:07.290460    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:08.060051 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:08.060016    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:08.708566 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:08.708519    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:09.604203 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:09.604166    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:10.014675 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:10:10.014632    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:10:10.334948 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:10.334860    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:11.010969 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:11.010922    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:11.699260 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:11.699223    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:12.400900 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:12.400857    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:13.110975 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:13.110930    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:13.875075 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:13.875036    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:14.701903 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:14.701875    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:14.813930 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:14.813892    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:15.178099 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.178022    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"]
Nov 01 23:10:15.178419 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.178407    5057 cpu_manager.go:414] "containerMap: removing container" podUID="afc8d5a4-4da5-48a3-a064-82ab972a3fb6" containerName="revoke-keys"
Nov 01 23:10:15.178463 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.178420    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="afc8d5a4-4da5-48a3-a064-82ab972a3fb6" containerName="revoke-keys"
Nov 01 23:10:15.178542 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.178532    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="afc8d5a4-4da5-48a3-a064-82ab972a3fb6" containerName="revoke-keys"
Nov 01 23:10:15.181561 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.181546    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.187275 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.187248    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 23:10:15.187427 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.187288    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m55gc\""
Nov 01 23:10:15.189582 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.189468    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"]
Nov 01 23:10:15.324795 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.324757    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skgf9\" (UniqueName: \"kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.325001 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.324881    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.325001 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.324914    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.425581 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.425528    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.425834 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.425613    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skgf9\" (UniqueName: \"kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.425834 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.425693    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.426301 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.426271    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.428092 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.428063    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.435053 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.434994    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgf9\" (UniqueName: \"kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9\") pod \"maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.496873 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.496839    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:15.543784 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.543721    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:15.779349 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.779265    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"]
Nov 01 23:10:15.781923 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:10:15.781894    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc97c3a6_9262_4f76_829a_7ab825f92a96.slice/crio-01c007afec7743630b77a9d648b273ec6f97e8273380820d2c1b223ba7ed8a4a WatchSource:0}: Error finding container 01c007afec7743630b77a9d648b273ec6f97e8273380820d2c1b223ba7ed8a4a: Status 404 returned error can't find the container with id 01c007afec7743630b77a9d648b273ec6f97e8273380820d2c1b223ba7ed8a4a
Nov 01 23:10:15.874634 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:15.874593    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb" event={"ID":"cc97c3a6-9262-4f76-829a-7ab825f92a96","Type":"ContainerStarted","Data":"01c007afec7743630b77a9d648b273ec6f97e8273380820d2c1b223ba7ed8a4a"}
Nov 01 23:10:16.435081 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:16.434983    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:16.881945 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:16.881911    5057 generic.go:356] "Generic (PLEG): container finished" podID="cc97c3a6-9262-4f76-829a-7ab825f92a96" containerID="05adcb7ba61230d46790063e395c528f78c4203e5127314d040557578e884af1" exitCode=0
Nov 01 23:10:16.882372 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:16.882006    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb" event={"ID":"cc97c3a6-9262-4f76-829a-7ab825f92a96","Type":"ContainerDied","Data":"05adcb7ba61230d46790063e395c528f78c4203e5127314d040557578e884af1"}
Nov 01 23:10:17.156217 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:17.156075    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:17.899970 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:17.899937    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:18.040467 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.040438    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:18.152284 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.152190    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9\" (UniqueName: \"kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9\") pod \"cc97c3a6-9262-4f76-829a-7ab825f92a96\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") "
Nov 01 23:10:18.152454 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.152307    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca\") pod \"cc97c3a6-9262-4f76-829a-7ab825f92a96\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") "
Nov 01 23:10:18.152454 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.152341    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca\") pod \"cc97c3a6-9262-4f76-829a-7ab825f92a96\" (UID: \"cc97c3a6-9262-4f76-829a-7ab825f92a96\") "
Nov 01 23:10:18.152741 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.152712    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca" pod "cc97c3a6-9262-4f76-829a-7ab825f92a96" (UID: "cc97c3a6-9262-4f76-829a-7ab825f92a96"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:10:18.154588 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.154562    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9" pod "cc97c3a6-9262-4f76-829a-7ab825f92a96" (UID: "cc97c3a6-9262-4f76-829a-7ab825f92a96"). InnerVolumeSpecName "kube-api-access-skgf9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:10:18.154681 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.154625    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca" pod "cc97c3a6-9262-4f76-829a-7ab825f92a96" (UID: "cc97c3a6-9262-4f76-829a-7ab825f92a96"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:10:18.253996 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.253955    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skgf9\" (UniqueName: \"kubernetes.io/projected/cc97c3a6-9262-4f76-829a-7ab825f92a96-kube-api-access-skgf9\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:10:18.253996 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.253988    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-openshift-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:10:18.253996 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.253999    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/cc97c3a6-9262-4f76-829a-7ab825f92a96-maas-api-xks-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:10:18.615936 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.615900    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:18.893013 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.892919    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"
Nov 01 23:10:18.893013 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.892950    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb" event={"ID":"cc97c3a6-9262-4f76-829a-7ab825f92a96","Type":"ContainerDied","Data":"01c007afec7743630b77a9d648b273ec6f97e8273380820d2c1b223ba7ed8a4a"}
Nov 01 23:10:18.893013 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:18.893006    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01c007afec7743630b77a9d648b273ec6f97e8273380820d2c1b223ba7ed8a4a"
Nov 01 23:10:19.215235 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:19.215195    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:19.895510 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:19.895462    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:20.590233 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:20.590197    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:21.284149 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:21.284109    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:21.983599 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:21.983557    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:22.713452 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:22.713412    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:23.400042 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:23.400009    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:24.074777 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:24.074742    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:24.761593 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:24.761552    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:25.404484 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:25.404435    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:25.531763 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:25.531723    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:25.687984 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:25.687953    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:31.350225 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:31.350192    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:31.481509 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:31.481457    5057 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"]
Nov 01 23:10:31.483244 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:31.483216    5057 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-authcfg-http-4b45d6-hh2lb"]
Nov 01 23:10:31.492158 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:31.492122    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:31.609404 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:31.609307    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:32.004740 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:32.004703    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:32.785290 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:32.785251    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:10:33.396100 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:33.396063    5057 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc97c3a6-9262-4f76-829a-7ab825f92a96" path="/var/lib/kubelet/pods/cc97c3a6-9262-4f76-829a-7ab825f92a96/volumes"
Nov 01 23:10:38.472635 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:38.472586    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:38.661067 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:38.661028    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:38.821032 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:38.820911    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:10:40.014695 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:10:40.014655    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:10:40.720463 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:40.720424    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:10:45.810361 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:45.810326    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:10:50.244676 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.244635    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"]
Nov 01 23:10:50.245096 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.245069    5057 cpu_manager.go:414] "containerMap: removing container" podUID="cc97c3a6-9262-4f76-829a-7ab825f92a96" containerName="revoke-keys"
Nov 01 23:10:50.245096 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.245080    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc97c3a6-9262-4f76-829a-7ab825f92a96" containerName="revoke-keys"
Nov 01 23:10:50.245183 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.245156    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc97c3a6-9262-4f76-829a-7ab825f92a96" containerName="revoke-keys"
Nov 01 23:10:50.249610 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.249589    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.254259 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.254219    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 23:10:50.254389 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.254348    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m55gc\""
Nov 01 23:10:50.256842 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.256816    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"]
Nov 01 23:10:50.410980 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.410932    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.411188 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.411079    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.411188 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.411155    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frzbc\" (UniqueName: \"kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.511742 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.511654    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.511742 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.511732    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.511932 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.511785    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frzbc\" (UniqueName: \"kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.512434 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.512405    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.514195 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.514167    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.520102 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.520079    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frzbc\" (UniqueName: \"kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc\") pod \"maas-api-revoke-sub-f03629db-nr8bf\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.564237 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.564193    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:50.838950 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:50.838923    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"]
Nov 01 23:10:50.841318 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:10:50.841289    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e7c4e8_78d0_4fb5_9476_0c035157ba7a.slice/crio-e90c3864baf22978c0caaa1a9de006ae9cade3203f941b7f813bf46459c218f9 WatchSource:0}: Error finding container e90c3864baf22978c0caaa1a9de006ae9cade3203f941b7f813bf46459c218f9: Status 404 returned error can't find the container with id e90c3864baf22978c0caaa1a9de006ae9cade3203f941b7f813bf46459c218f9
Nov 01 23:10:51.101610 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:51.101521    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf" event={"ID":"52e7c4e8-78d0-4fb5-9476-0c035157ba7a","Type":"ContainerStarted","Data":"e90c3864baf22978c0caaa1a9de006ae9cade3203f941b7f813bf46459c218f9"}
Nov 01 23:10:52.107817 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:52.107781    5057 generic.go:356] "Generic (PLEG): container finished" podID="52e7c4e8-78d0-4fb5-9476-0c035157ba7a" containerID="f65b24218d5e19e6b615313c9c96faa392ab444608a757fbcdf68fa993eb590c" exitCode=0
Nov 01 23:10:52.108256 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:52.107842    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf" event={"ID":"52e7c4e8-78d0-4fb5-9476-0c035157ba7a","Type":"ContainerDied","Data":"f65b24218d5e19e6b615313c9c96faa392ab444608a757fbcdf68fa993eb590c"}
Nov 01 23:10:53.250873 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.250843    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:53.339417 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.339378    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc\" (UniqueName: \"kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc\") pod \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") "
Nov 01 23:10:53.339417 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.339422    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca\") pod \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") "
Nov 01 23:10:53.339727 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.339487    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca\") pod \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\" (UID: \"52e7c4e8-78d0-4fb5-9476-0c035157ba7a\") "
Nov 01 23:10:53.339952 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.339927    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca" pod "52e7c4e8-78d0-4fb5-9476-0c035157ba7a" (UID: "52e7c4e8-78d0-4fb5-9476-0c035157ba7a"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:10:53.341799 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.341778    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc" pod "52e7c4e8-78d0-4fb5-9476-0c035157ba7a" (UID: "52e7c4e8-78d0-4fb5-9476-0c035157ba7a"). InnerVolumeSpecName "kube-api-access-frzbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:10:53.341877 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.341851    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca" pod "52e7c4e8-78d0-4fb5-9476-0c035157ba7a" (UID: "52e7c4e8-78d0-4fb5-9476-0c035157ba7a"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:10:53.440762 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.440726    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frzbc\" (UniqueName: \"kubernetes.io/projected/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-kube-api-access-frzbc\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:10:53.440762 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.440759    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-xks-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:10:53.440762 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:53.440769    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/52e7c4e8-78d0-4fb5-9476-0c035157ba7a-maas-api-openshift-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:10:54.118903 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:54.118871    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf"
Nov 01 23:10:54.119094 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:54.118870    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-nr8bf" event={"ID":"52e7c4e8-78d0-4fb5-9476-0c035157ba7a","Type":"ContainerDied","Data":"e90c3864baf22978c0caaa1a9de006ae9cade3203f941b7f813bf46459c218f9"}
Nov 01 23:10:54.119094 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:10:54.119035    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e90c3864baf22978c0caaa1a9de006ae9cade3203f941b7f813bf46459c218f9"
Nov 01 23:11:01.199852 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:01.199813    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:11:05.564674 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:05.564622    5057 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-58847f9fc5-9zrmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Nov 01 23:11:05.565410 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:05.565371    5057 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-58847f9fc5-9zrmc" podUID="73c13087-b188-46c3-a94c-68b8b8b81448" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.115:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Nov 01 23:11:06.444054 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.444016    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"]
Nov 01 23:11:06.444419 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.444407    5057 cpu_manager.go:414] "containerMap: removing container" podUID="52e7c4e8-78d0-4fb5-9476-0c035157ba7a" containerName="revoke-keys"
Nov 01 23:11:06.444466 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.444420    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="52e7c4e8-78d0-4fb5-9476-0c035157ba7a" containerName="revoke-keys"
Nov 01 23:11:06.444544 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.444530    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="52e7c4e8-78d0-4fb5-9476-0c035157ba7a" containerName="revoke-keys"
Nov 01 23:11:06.447860 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.447843    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.453781 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.453754    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 23:11:06.455290 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.455259    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m55gc\""
Nov 01 23:11:06.457437 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.457412    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"]
Nov 01 23:11:06.489319 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.489272    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.489508 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.489341    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8vdj\" (UniqueName: \"kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.489508 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.489393    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.590989 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.590947    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.591520 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.591061    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.591520 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.591096    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8vdj\" (UniqueName: \"kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.591782 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.591762    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.593476 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.593453    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.599830 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.599808    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8vdj\" (UniqueName: \"kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj\") pod \"maas-api-revoke-sub-0165d7d2-x8mvk\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:06.762103 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:06.762060    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:07.037222 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:07.037146    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"]
Nov 01 23:11:07.040591 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:11:07.040552    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca755f30_1eff_4630_9dde_24e6b420a00d.slice/crio-76fc577e8379d8f2e0e45813163eb25e90133f8f10ca08b8de4d7f8a7803b289 WatchSource:0}: Error finding container 76fc577e8379d8f2e0e45813163eb25e90133f8f10ca08b8de4d7f8a7803b289: Status 404 returned error can't find the container with id 76fc577e8379d8f2e0e45813163eb25e90133f8f10ca08b8de4d7f8a7803b289
Nov 01 23:11:07.213366 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:07.213327    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk" event={"ID":"ca755f30-1eff-4630-9dde-24e6b420a00d","Type":"ContainerStarted","Data":"76fc577e8379d8f2e0e45813163eb25e90133f8f10ca08b8de4d7f8a7803b289"}
Nov 01 23:11:08.224566 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:08.224526    5057 generic.go:356] "Generic (PLEG): container finished" podID="ca755f30-1eff-4630-9dde-24e6b420a00d" containerID="1bc5cb9605c2e1ccd3c1a1272408c788b78e11b11b928a2ad8bc9df40a633b1d" exitCode=0
Nov 01 23:11:08.224982 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:08.224616    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk" event={"ID":"ca755f30-1eff-4630-9dde-24e6b420a00d","Type":"ContainerDied","Data":"1bc5cb9605c2e1ccd3c1a1272408c788b78e11b11b928a2ad8bc9df40a633b1d"}
Nov 01 23:11:09.375437 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.375409    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:09.419120 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.419088    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj\" (UniqueName: \"kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj\") pod \"ca755f30-1eff-4630-9dde-24e6b420a00d\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") "
Nov 01 23:11:09.419314 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.419195    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca\") pod \"ca755f30-1eff-4630-9dde-24e6b420a00d\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") "
Nov 01 23:11:09.419314 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.419233    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca\") pod \"ca755f30-1eff-4630-9dde-24e6b420a00d\" (UID: \"ca755f30-1eff-4630-9dde-24e6b420a00d\") "
Nov 01 23:11:09.419721 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.419687    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca" pod "ca755f30-1eff-4630-9dde-24e6b420a00d" (UID: "ca755f30-1eff-4630-9dde-24e6b420a00d"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:11:09.421420 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.421390    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj" pod "ca755f30-1eff-4630-9dde-24e6b420a00d" (UID: "ca755f30-1eff-4630-9dde-24e6b420a00d"). InnerVolumeSpecName "kube-api-access-g8vdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:11:09.421542 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.421410    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca" pod "ca755f30-1eff-4630-9dde-24e6b420a00d" (UID: "ca755f30-1eff-4630-9dde-24e6b420a00d"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:11:09.521037 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.520959    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8vdj\" (UniqueName: \"kubernetes.io/projected/ca755f30-1eff-4630-9dde-24e6b420a00d-kube-api-access-g8vdj\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:11:09.521037 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.520992    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-xks-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:11:09.521037 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:09.521002    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/ca755f30-1eff-4630-9dde-24e6b420a00d-maas-api-openshift-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:11:10.014034 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:11:10.013992    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:11:10.235667 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:10.235635    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk"
Nov 01 23:11:10.235852 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:10.235674    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-x8mvk" event={"ID":"ca755f30-1eff-4630-9dde-24e6b420a00d","Type":"ContainerDied","Data":"76fc577e8379d8f2e0e45813163eb25e90133f8f10ca08b8de4d7f8a7803b289"}
Nov 01 23:11:10.235852 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:10.235728    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76fc577e8379d8f2e0e45813163eb25e90133f8f10ca08b8de4d7f8a7803b289"
Nov 01 23:11:14.845752 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:14.845717    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:11:17.229877 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.229836    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"]
Nov 01 23:11:17.230310 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.230296    5057 cpu_manager.go:414] "containerMap: removing container" podUID="ca755f30-1eff-4630-9dde-24e6b420a00d" containerName="revoke-keys"
Nov 01 23:11:17.230356 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.230311    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ca755f30-1eff-4630-9dde-24e6b420a00d" containerName="revoke-keys"
Nov 01 23:11:17.230391 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.230375    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="ca755f30-1eff-4630-9dde-24e6b420a00d" containerName="revoke-keys"
Nov 01 23:11:17.233802 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.233779    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.241941 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.241915    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\""
Nov 01 23:11:17.242066 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.241991    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-m55gc\""
Nov 01 23:11:17.253631 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.253592    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"]
Nov 01 23:11:17.294617 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.294582    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qtkd\" (UniqueName: \"kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.294807 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.294642    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.294807 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.294738    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.395831 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.395800    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.396034 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.395841    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.396034 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.395906    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qtkd\" (UniqueName: \"kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.396636 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.396611    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.398241 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.398217    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.407979 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.407955    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qtkd\" (UniqueName: \"kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd\") pod \"maas-api-revoke-sub-6b19338f-wjb92\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.547649 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.547559    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:17.833135 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:17.833034    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"]
Nov 01 23:11:17.841747 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:11:17.841717    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5ce088d_d9a6_4c1a_95f5_44d1517a4bf8.slice/crio-ed9a9ec7077876569191acd2f9af731d91893b879dfd6fb2b39bdd29d1899e62 WatchSource:0}: Error finding container ed9a9ec7077876569191acd2f9af731d91893b879dfd6fb2b39bdd29d1899e62: Status 404 returned error can't find the container with id ed9a9ec7077876569191acd2f9af731d91893b879dfd6fb2b39bdd29d1899e62
Nov 01 23:11:18.279789 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:18.279737    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92" event={"ID":"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8","Type":"ContainerStarted","Data":"98f8bfafd93c09cebbea5d5b04a9ce5ce46ee0d36483252a1b44e167afd225e8"}
Nov 01 23:11:18.280213 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:18.279810    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92" event={"ID":"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8","Type":"ContainerStarted","Data":"ed9a9ec7077876569191acd2f9af731d91893b879dfd6fb2b39bdd29d1899e62"}
Nov 01 23:11:18.303760 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:18.303700    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92" podStartSLOduration=1.303686356 podStartE2EDuration="1.303686356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-01 23:11:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 23:11:18.302352377 +0000 UTC m=+2935.044047996" watchObservedRunningTime="2026-11-01 23:11:18.303686356 +0000 UTC m=+2935.045381975"
Nov 01 23:11:19.285703 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:19.285661    5057 generic.go:356] "Generic (PLEG): container finished" podID="f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" containerID="98f8bfafd93c09cebbea5d5b04a9ce5ce46ee0d36483252a1b44e167afd225e8" exitCode=0
Nov 01 23:11:19.286107 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:19.285737    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92" event={"ID":"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8","Type":"ContainerDied","Data":"98f8bfafd93c09cebbea5d5b04a9ce5ce46ee0d36483252a1b44e167afd225e8"}
Nov 01 23:11:20.431846 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.431815    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:20.526309 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.526274    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca\") pod \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") "
Nov 01 23:11:20.526536 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.526325    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca\") pod \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") "
Nov 01 23:11:20.526536 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.526384    5057 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd\" (UniqueName: \"kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd\") pod \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\" (UID: \"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8\") "
Nov 01 23:11:20.526784 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.526761    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca" pod "f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" (UID: "f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Nov 01 23:11:20.528535 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.528509    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca" pod "f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" (UID: "f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Nov 01 23:11:20.528651 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.528581    5057 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd" pod "f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" (UID: "f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8"). InnerVolumeSpecName "kube-api-access-5qtkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Nov 01 23:11:20.627921 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.627836    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-xks-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:11:20.627921 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.627868    5057 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-maas-api-openshift-ca\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:11:20.627921 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:20.627879    5057 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qtkd\" (UniqueName: \"kubernetes.io/projected/f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8-kube-api-access-5qtkd\") on node \"ip-10-0-141-148.us-east-2.compute.internal\" DevicePath \"\""
Nov 01 23:11:21.297455 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:21.297422    5057 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92"
Nov 01 23:11:21.297659 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:21.297424    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-wjb92" event={"ID":"f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8","Type":"ContainerDied","Data":"ed9a9ec7077876569191acd2f9af731d91893b879dfd6fb2b39bdd29d1899e62"}
Nov 01 23:11:21.297659 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:21.297560    5057 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed9a9ec7077876569191acd2f9af731d91893b879dfd6fb2b39bdd29d1899e62"
Nov 01 23:11:28.181748 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:28.181706    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-spsmr"]
Nov 01 23:11:38.897767 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:38.897731    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-sc9w7_9bca932e-0124-45e7-8879-2df04b4706da/manager/0.log"
Nov 01 23:11:39.700828 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:39.700792    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-lj699_0983b1d5-dfad-454e-b71a-be502461bd6b/proxy/0.log"
Nov 01 23:11:40.014906 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:11:40.014819    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:11:40.955625 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:40.955590    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-x8mvk_ca755f30-1eff-4630-9dde-24e6b420a00d/revoke-keys/0.log"
Nov 01 23:11:41.442750 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:41.442671    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-wjb92_f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8/revoke-keys/0.log"
Nov 01 23:11:41.759567 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:41.759537    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-nr8bf_52e7c4e8-78d0-4fb5-9476-0c035157ba7a/revoke-keys/0.log"
Nov 01 23:11:41.928221 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:41.928190    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-455hj_df79818f-3c33-4b20-8c29-929ae609823d/postgres/0.log"
Nov 01 23:11:43.528349 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:43.528310    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vtm7p_c3b1f1cd-52ff-4efe-872f-7b767f340e58/manager/0.log"
Nov 01 23:11:44.079438 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:44.079404    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-5h446_a0096d7f-77fb-406d-ab0d-6ee3f70f007f/manager/0.log"
Nov 01 23:11:44.289553 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:44.289522    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-spsmr_1b60af0c-bab8-4187-a0c2-f64f8ae42fa3/limitador/0.log"
Nov 01 23:11:45.377181 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:45.377148    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45_a31555b0-ec20-4c38-a4e0-4235abb76d9a/istio-proxy/0.log"
Nov 01 23:11:46.404051 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:46.404017    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-9zrmc_73c13087-b188-46c3-a94c-68b8b8b81448/istio-proxy/0.log"
Nov 01 23:11:46.583443 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:46.583395    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-d6k7r_a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec/istio-proxy/0.log"
Nov 01 23:11:46.907195 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:46.907163    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/1.log"
Nov 01 23:11:47.761337 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:47.761286    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6_c27e9d85-ba64-411d-9423-b3819825ea02/istio-proxy/0.log"
Nov 01 23:11:48.266786 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:48.266753    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8444bb9d4c-m4lpd_ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/router/0.log"
Nov 01 23:11:48.848795 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:11:48.848763    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/0.log"
Nov 01 23:12:00.780580 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:00.780547    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-sc9w7_9bca932e-0124-45e7-8879-2df04b4706da/manager/0.log"
Nov 01 23:12:01.548513 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:01.548453    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-lj699_0983b1d5-dfad-454e-b71a-be502461bd6b/proxy/0.log"
Nov 01 23:12:02.408322 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:02.408291    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-x8mvk_ca755f30-1eff-4630-9dde-24e6b420a00d/revoke-keys/0.log"
Nov 01 23:12:02.916205 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:02.916177    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-wjb92_f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8/revoke-keys/0.log"
Nov 01 23:12:03.249312 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:03.249283    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-nr8bf_52e7c4e8-78d0-4fb5-9476-0c035157ba7a/revoke-keys/0.log"
Nov 01 23:12:03.421465 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:03.421428    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-455hj_df79818f-3c33-4b20-8c29-929ae609823d/postgres/0.log"
Nov 01 23:12:04.728465 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:04.728430    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vtm7p_c3b1f1cd-52ff-4efe-872f-7b767f340e58/manager/0.log"
Nov 01 23:12:05.233305 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:05.233266    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-5h446_a0096d7f-77fb-406d-ab0d-6ee3f70f007f/manager/0.log"
Nov 01 23:12:05.421033 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:05.420999    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-spsmr_1b60af0c-bab8-4187-a0c2-f64f8ae42fa3/limitador/0.log"
Nov 01 23:12:06.076819 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:06.076784    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45_a31555b0-ec20-4c38-a4e0-4235abb76d9a/istio-proxy/0.log"
Nov 01 23:12:07.108146 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:07.108114    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-9zrmc_73c13087-b188-46c3-a94c-68b8b8b81448/istio-proxy/0.log"
Nov 01 23:12:07.274933 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:07.274899    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-d6k7r_a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec/istio-proxy/0.log"
Nov 01 23:12:07.627658 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:07.627622    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/1.log"
Nov 01 23:12:08.437937 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:08.437909    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6_c27e9d85-ba64-411d-9423-b3819825ea02/istio-proxy/0.log"
Nov 01 23:12:08.947063 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:08.947019    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8444bb9d4c-m4lpd_ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/router/0.log"
Nov 01 23:12:10.015088 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:12:10.015048    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:12:14.878712 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:14.878655    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:12:23.850911 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:23.850880    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:12:23.869687 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:12:23.869660    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:12:40.014025 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:12:40.013943    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:13:01.377214 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:01.377147    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:13:10.014547 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:13:10.014505    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:13:14.912751 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:14.912714    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:13:22.680651 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:22.680605    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:13:22.845345 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:22.845294    5057 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log" limit=1024
Nov 01 23:13:26.370306 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:26.370259    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9s2pz_304f876e-02df-45da-af55-62cae8468a50/global-pull-secret-syncer/0.log"
Nov 01 23:13:26.504657 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:26.504620    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-lkzkr_42fd8ec3-88cb-4e0b-9d58-1c654ff20582/connection-checker/0.log"
Nov 01 23:13:26.530609 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:26.530585    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-r6xqj_514791bf-52ab-46d1-bec2-44baa1b3f48d/connection-checker/0.log"
Nov 01 23:13:26.556581 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:26.556555    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-rxq5k_1e3cf582-d2fe-4d46-8945-e8b1bfe6bf0d/connection-checker/0.log"
Nov 01 23:13:26.622186 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:26.622093    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-b9n6q_c80d6c27-cb56-46c7-bed5-b96e23274d9c/konnectivity-agent/0.log"
Nov 01 23:13:26.816458 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:26.816422    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-148.us-east-2.compute.internal_68cec1540c43f1cf2055316a4f5734e4/haproxy/0.log"
Nov 01 23:13:32.998727 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:32.998689    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-7b7bb4bc77-vtm7p_c3b1f1cd-52ff-4efe-872f-7b767f340e58/manager/0.log"
Nov 01 23:13:33.133450 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:33.133410    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-678d87766f-5h446_a0096d7f-77fb-406d-ab0d-6ee3f70f007f/manager/0.log"
Nov 01 23:13:33.162580 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:33.162550    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-spsmr_1b60af0c-bab8-4187-a0c2-f64f8ae42fa3/limitador/0.log"
Nov 01 23:13:34.892161 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:34.892126    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-l4759_71d83910-eb9d-4178-b1c3-3fdf24524dea/cluster-monitoring-operator/0.log"
Nov 01 23:13:35.238688 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:35.238658    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sb596_ed7e9b70-7315-4c81-8073-d5285edb29d7/node-exporter/0.log"
Nov 01 23:13:35.261057 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:35.261028    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sb596_ed7e9b70-7315-4c81-8073-d5285edb29d7/kube-rbac-proxy/0.log"
Nov 01 23:13:35.284082 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:35.284061    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-sb596_ed7e9b70-7315-4c81-8073-d5285edb29d7/init-textfile/0.log"
Nov 01 23:13:35.675906 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:35.675819    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-6489d8bf89-tc5r8_8847c4ae-f454-4ec5-99dd-3279b6753bc5/prometheus-operator-admission-webhook/0.log"
Nov 01 23:13:36.346946 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.346913    5057 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"]
Nov 01 23:13:36.347342 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.347328    5057 cpu_manager.go:414] "containerMap: removing container" podUID="f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" containerName="revoke-keys"
Nov 01 23:13:36.347385 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.347343    5057 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" containerName="revoke-keys"
Nov 01 23:13:36.347436 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.347427    5057 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5ce088d-d9a6-4c1a-95f5-44d1517a4bf8" containerName="revoke-keys"
Nov 01 23:13:36.350471 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.350453    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.357139 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.357118    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w64nw\"/\"openshift-service-ca.crt\""
Nov 01 23:13:36.358053 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.358039    5057 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w64nw\"/\"kube-root-ca.crt\""
Nov 01 23:13:36.358130 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.358119    5057 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w64nw\"/\"default-dockercfg-nlr9s\""
Nov 01 23:13:36.360139 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.360117    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"]
Nov 01 23:13:36.436850 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.436808    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9528\" (UniqueName: \"kubernetes.io/projected/b92c634c-ba1f-489b-945a-9776f8fc3cd0-kube-api-access-k9528\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.437042 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.436875    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-sys\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.437042 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.436974    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-podres\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.437042 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.437027    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-lib-modules\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.437167 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.437142    5057 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-proc\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538697 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538658    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-proc\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538706    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9528\" (UniqueName: \"kubernetes.io/projected/b92c634c-ba1f-489b-945a-9776f8fc3cd0-kube-api-access-k9528\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538746    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-sys\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538775    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-podres\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538802    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-proc\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538813    5057 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-lib-modules\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538869    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-sys\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538880    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-podres\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.538919 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.538879    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b92c634c-ba1f-489b-945a-9776f8fc3cd0-lib-modules\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.547319 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.547291    5057 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9528\" (UniqueName: \"kubernetes.io/projected/b92c634c-ba1f-489b-945a-9776f8fc3cd0-kube-api-access-k9528\") pod \"perf-node-gather-daemonset-j6fzg\" (UID: \"b92c634c-ba1f-489b-945a-9776f8fc3cd0\") " pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.664653 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.664553    5057 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:36.940485 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.940457    5057 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"]
Nov 01 23:13:36.942931 ip-10-0-141-148 kubenswrapper[5057]: W1101 23:13:36.942900    5057 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb92c634c_ba1f_489b_945a_9776f8fc3cd0.slice/crio-44ef86fa1365cf4349f7961608236af9ff55dec6f09b314bb0284fd99566883d WatchSource:0}: Error finding container 44ef86fa1365cf4349f7961608236af9ff55dec6f09b314bb0284fd99566883d: Status 404 returned error can't find the container with id 44ef86fa1365cf4349f7961608236af9ff55dec6f09b314bb0284fd99566883d
Nov 01 23:13:36.990246 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:36.990216    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg" event={"ID":"b92c634c-ba1f-489b-945a-9776f8fc3cd0","Type":"ContainerStarted","Data":"44ef86fa1365cf4349f7961608236af9ff55dec6f09b314bb0284fd99566883d"}
Nov 01 23:13:37.171688 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:37.171658    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-86c478bb88-nk6b8_9918792e-83ee-41de-8794-bf799e14788d/networking-console-plugin/0.log"
Nov 01 23:13:37.829579 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:37.829539    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/1.log"
Nov 01 23:13:37.838322 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:37.838284    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-zz5p7_812cb630-c9f1-4b19-b5e2-b57fda496558/console-operator/2.log"
Nov 01 23:13:37.996819 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:37.996781    5057 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg" event={"ID":"b92c634c-ba1f-489b-945a-9776f8fc3cd0","Type":"ContainerStarted","Data":"12c83957e70f412398972e9bab6035d70926e6769afcfb440c2bbbfc36198a58"}
Nov 01 23:13:37.997005 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:37.996980    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:38.013031 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:38.012979    5057 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg" podStartSLOduration=2.012965708 podStartE2EDuration="2.012965708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-01 23:13:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-01 23:13:38.010954719 +0000 UTC m=+3074.752650319" watchObservedRunningTime="2026-11-01 23:13:38.012965708 +0000 UTC m=+3074.754661327"
Nov 01 23:13:38.383545 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:38.383485    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-6dd789c8b7-ghc6x_08dc2d62-7a41-408a-a1cf-0b291f633f2f/download-server/0.log"
Nov 01 23:13:38.979565 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:38.979521    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-58c9f5f46d-pnqc5_c6a49d28-717c-4fab-91be-c77a2e58b037/volume-data-source-validator/0.log"
Nov 01 23:13:40.014068 ip-10-0-141-148 kubenswrapper[5057]: E1101 23:13:40.014033    5057 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-f54gq" containerName="extractor"
Nov 01 23:13:40.134288 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:40.134256    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fzbm2_81511c2e-f1cb-40ea-8d52-b795feb55690/dns/0.log"
Nov 01 23:13:40.161536 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:40.161507    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-fzbm2_81511c2e-f1cb-40ea-8d52-b795feb55690/kube-rbac-proxy/0.log"
Nov 01 23:13:40.393030 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:40.392945    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9xfht_295d3d68-c2d1-497f-9a6b-7bd5017b2a41/dns-node-resolver/0.log"
Nov 01 23:13:41.091385 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:41.091343    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-79448f7f6f-r8bjp_8d8cd38e-474f-47aa-ba46-fe09d424bf53/registry/0.log"
Nov 01 23:13:41.143852 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:41.143821    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-vjxrr_4e7fb00c-4f60-4b4c-9402-65cc6ec7e887/node-ca/0.log"
Nov 01 23:13:42.209433 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:42.209403    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45_a31555b0-ec20-4c38-a4e0-4235abb76d9a/istio-proxy/0.log"
Nov 01 23:13:44.012405 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:44.012377    5057 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w64nw/perf-node-gather-daemonset-j6fzg"
Nov 01 23:13:45.450945 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:45.450915    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-9zrmc_73c13087-b188-46c3-a94c-68b8b8b81448/istio-proxy/0.log"
Nov 01 23:13:45.493385 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:45.493352    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-d6k7r_a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec/istio-proxy/0.log"
Nov 01 23:13:45.552503 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:45.552465    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/1.log"
Nov 01 23:13:45.552766 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:45.552612    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/0.log"
Nov 01 23:13:45.730848 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:45.730747    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6_c27e9d85-ba64-411d-9423-b3819825ea02/istio-proxy/0.log"
Nov 01 23:13:45.836785 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:45.836749    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8444bb9d4c-m4lpd_ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/router/0.log"
Nov 01 23:13:46.899236 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:46.899208    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45_a31555b0-ec20-4c38-a4e0-4235abb76d9a/istio-proxy/0.log"
Nov 01 23:13:48.077433 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.077402    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-j85jx_cc378d17-492d-4255-bd20-998d518e8b04/serve-healthcheck-canary/0.log"
Nov 01 23:13:48.524689 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.524651    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-fvxj8_d91a8683-96c3-4725-b90b-da6590c5076e/insights-operator/1.log"
Nov 01 23:13:48.528035 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.528002    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-7c4986d449-fvxj8_d91a8683-96c3-4725-b90b-da6590c5076e/insights-operator/2.log"
Nov 01 23:13:48.633616 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.633586    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f54gq_cb04c807-ee72-4833-a98b-689f42b18e4c/kube-rbac-proxy/0.log"
Nov 01 23:13:48.657476 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.657445    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f54gq_cb04c807-ee72-4833-a98b-689f42b18e4c/exporter/0.log"
Nov 01 23:13:48.682104 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.682074    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-f54gq_cb04c807-ee72-4833-a98b-689f42b18e4c/extractor/0.log"
Nov 01 23:13:48.815833 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.815742    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-kbdg8-jm4bc_5214431e-5fc7-4ec2-af7c-e32a789c7d0f/insights-gathering/0.log"
Nov 01 23:13:48.854137 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:48.854064    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_periodic-gathering-qsqdj-ldvb2_3977575c-5b80-4a00-b1bb-310cb902b2be/insights-gathering/0.log"
Nov 01 23:13:49.731881 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:49.731852    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-9zrmc_73c13087-b188-46c3-a94c-68b8b8b81448/istio-proxy/0.log"
Nov 01 23:13:49.753457 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:49.753425    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-d6k7r_a2f080e5-a870-4c58-9f4e-2ba21bf7a0ec/istio-proxy/0.log"
Nov 01 23:13:49.801020 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:49.800989    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/1.log"
Nov 01 23:13:49.801218 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:49.801120    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-58847f9fc5-kbn9c_9a077e74-c5a6-4e89-8ff3-94ac4df0188e/istio-proxy/0.log"
Nov 01 23:13:49.879977 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:49.879938    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_openshift-ai-inference-openshift-ai-inference-856b6dffcf-nzgn6_c27e9d85-ba64-411d-9423-b3819825ea02/istio-proxy/0.log"
Nov 01 23:13:49.933144 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:49.933115    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-8444bb9d4c-m4lpd_ac40f60c-ca7d-48cf-92fe-a8fea30a5d54/router/0.log"
Nov 01 23:13:51.268209 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:51.268177    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8487c6b6d6-f98qg_d74ad49a-0cce-4daa-a991-0e65c6c24765/manager/0.log"
Nov 01 23:13:51.293750 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:51.293727    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub-operator-system_opendatahub-operator-controller-manager-8487c6b6d6-f98qg_d74ad49a-0cce-4daa-a991-0e65c6c24765/copy-manifests/0.log"
Nov 01 23:13:52.696754 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:52.696725    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-jobset-operator_jobset-operator-674cd59dc5-jxxbm_86b517fe-edbf-4182-803d-f61b65a6eb09/jobset-operator/0.log"
Nov 01 23:13:52.834520 ip-10-0-141-148 kubenswrapper[5057]: I1101 23:13:52.834464    5057 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-59cf4dc6bfwwx45_a31555b0-ec20-4c38-a4e0-4235abb76d9a/istio-proxy/0.log"